-- Logs begin at Wed 2022-08-24 15:53:26 UTC, end at Wed 2022-08-24 17:15:04 UTC. -- Aug 24 16:26:37.924440 dv-cinder-852208-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 16:26:37.937993 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 16:26:37.944531 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 16:26:37 2022] *** Aug 24 16:26:37.944531 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 16:26:37.944531 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 16:26:37.944531 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: nodename: dv-cinder-852208-9-iscsi Aug 24 16:26:37.944531 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: machine: x86_64 Aug 24 16:26:37.944531 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: clock source: unix Aug 24 16:26:37.944531 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: pcre jit disabled Aug 24 16:26:37.944531 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: detected number of CPU cores: 4 Aug 24 16:26:37.944531 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: current working directory: / Aug 24 16:26:37.945125 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: detected binary path: /usr/bin/uwsgi-core Aug 24 16:26:37.945125 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: your processes number limit is 47680 Aug 24 16:26:37.945125 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: your memory page size is 4096 bytes Aug 24 16:26:37.945125 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: detected max file descriptor number: 2048 Aug 24 16:26:37.945125 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: lock engine: pthread robust mutexes Aug 24 16:26:37.945125 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: thunder lock: enabled Aug 24 16:26:37.945125 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 16:26:37.945507 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 16:26:38.013092 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: Python main interpreter initialized at 0x563e7f0aa9c0 Aug 24 16:26:38.013092 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: python threads support enabled Aug 24 16:26:38.013092 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: your server socket listen backlog is limited to 100 connections Aug 24 16:26:38.013092 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: your mercy for graceful operations on workers is 90 seconds Aug 24 16:26:38.019257 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 16:26:38.019257 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: *** Operational MODE: preforking *** Aug 24 16:26:38.019257 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: *** uWSGI is running in multiple interpreter mode *** Aug 24 16:26:38.019257 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: spawned uWSGI master process (pid: 81827) Aug 24 16:26:38.019257 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: spawned uWSGI worker 1 (pid: 81843, cores: 1) Aug 24 16:26:38.019257 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: spawned uWSGI worker 2 (pid: 81844, cores: 1) Aug 24 16:26:38.019257 dv-cinder-852208-9-iscsi devstack@keystone.service[81827]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 16:26:38.014516 dv-cinder-852208-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 16:26:42.429324 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 16:26:42.429324 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: from cryptography.utils import int_from_bytes Aug 24 16:26:42.430573 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 16:26:42.430573 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: from cryptography.utils import int_from_bytes Aug 24 16:26:42.431279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 16:26:42.431279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: from cryptography.utils import int_from_bytes Aug 24 16:26:42.431502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 16:26:42.431502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: from cryptography.utils import int_from_bytes Aug 24 16:26:42.481698 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 16:26:42.481998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 16:26:42.482147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] command line args: [] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 16:26:42.482308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 16:26:42.482497 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ================================================================================ {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 16:26:42.482681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 16:26:42.482837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] admin_token = **** {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.482985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 16:26:42.483121 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] config_dir = [] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.483262 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] command line args: [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 16:26:42.483405 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.483559 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 16:26:42.483701 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] config_source = [] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.483832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ================================================================================ {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 16:26:42.483966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] debug = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.484108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] admin_token = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.484246 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.486810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] config_dir = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.486959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.487088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.487304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.487424 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] config_source = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.487424 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] debug = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.487662 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.487788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.488005 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.488133 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.488259 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] list_limit = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.488385 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.488542 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] log_config_append = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.488542 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.488761 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.488886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] log_dir = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.489013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.489129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] log_file = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.489246 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] list_limit = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.489372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.489488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] log_config_append = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.489615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.489740 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.489858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.489978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] log_dir = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.490091 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.490217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] log_file = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.490330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.490465 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.490578 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.490699 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.490818 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.490937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.491067 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.491205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.491317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.491433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.491552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.491670 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.491801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.491922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.492041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.492162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.492276 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.492394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.492562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.492674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.492797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.492907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.492907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.492907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.493233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.70/identity {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.493233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] publish_errors = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.493457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.493567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.493759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.493873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.493873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.493873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.494169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.70/identity {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.494169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] publish_errors = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.494388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.494507 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.494625 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] standard_threads = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.494780 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.494906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.495037 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.495152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.495264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.495394 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.495394 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] use_journal = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.495603 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.495722 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] use_json = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.495832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] standard_threads = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.495954 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] use_stderr = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.496077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.496191 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] use_syslog = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.496191 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.496389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.496389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.496976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.497229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] use_journal = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.497350 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.497479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] use_json = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.497479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] use_stderr = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.497691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.497803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] use_syslog = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.497932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.498127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:26:42.498297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.498508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.498644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.498834 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.498979 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.499118 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.499247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.499376 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.499519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.499611 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.499747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.499879 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.500011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.500137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.500265 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.500397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.500580 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.500710 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.500847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.500978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.501111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.501248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.501248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.501511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.501648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.501648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.501906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.502059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.502059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.502309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.502436 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.502568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.502696 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.502696 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.503018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.503147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.503147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.503391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.503663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.503913 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.504046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.504046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.504269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.504547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.504547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.504818 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.504950 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.505077 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.505211 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.505211 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.505456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.505601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.505726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.505853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.505853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.506087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.506216 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.506216 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.506462 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.506588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.506721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.506850 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.506850 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.507086 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.507185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.507311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.507441 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.507441 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.507672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.507856 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.507856 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.508183 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.508352 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.508606 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.508791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.508791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.509116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.509289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.509289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.509610 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.509757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.509870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.509970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.509970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.510268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.510402 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.510592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.510792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.510792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.511111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.511301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.511301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.511604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.511737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.511883 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.512068 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.512068 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.512361 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.512631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.512631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.512777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.512847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.512909 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.512969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.512969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.513086 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.513162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.513162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.513365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.513491 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.513491 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.513761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.513909 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.514101 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.514234 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.514234 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.514560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.514748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.514748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.515056 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.515252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.515387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.515581 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.515581 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.515847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.515932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.515932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.516193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.516580 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.516580 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.516908 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.517042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.517177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.517820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.517820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.518056 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.518260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.518260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.518497 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.518626 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.518626 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.518859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.519107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.519107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.519341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.519466 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.519594 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.519732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.519732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.519998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.520128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.520128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.520368 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.520586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.520586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.521005 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.521140 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.521281 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.521411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.521411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.521641 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.521781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.521910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.522021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.522021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.522328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.522454 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.522454 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.522700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.522825 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.522963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.523091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] auth.external = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.523091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.523399 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.523529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.523657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.523787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.523787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] auth.password = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.524022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.524162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] auth.token = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.524162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.524392 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.524548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.524669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.524790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.524790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.525041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.525166 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.525297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.525426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.525426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.525678 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.525809 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] credential.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.525809 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.526050 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.526193 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.526330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.526464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.526464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.526700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.526827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.526955 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.527082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.527082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.527327 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.527457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.527457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.527712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.527858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.528038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.528180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.528180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.528482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.528617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.528749 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.528879 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.528879 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.529141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.529277 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.529397 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.529534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.529534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.529838 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.529981 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.530114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.530242 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.530242 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.530495 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.530728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.530871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.531006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.531006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.531250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.531377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.531511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.531657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.531657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.531910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.532035 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.532150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.532371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.532371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.532669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.532802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.532802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.533623 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.533770 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.533963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.534097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.534097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.534334 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.534475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.534717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.534850 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.534850 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.535097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.535228 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.535228 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.535506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] auth.external = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.535638 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.535779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.535912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.535912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.536152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.536298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.536298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.536686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.536863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.537037 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] auth.password = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.537210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.537210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.537557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] auth.token = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.537732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.537903 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.538075 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.538075 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.538398 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.538577 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.538757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.538935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.538935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.539250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.539431 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.539599 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.539763 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.539763 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.540071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.540233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.540233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.540587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.540764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.540937 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] credential.caching = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.541103 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.541103 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.541563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.541735 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.541905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.542074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.542074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.542398 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.542579 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.542749 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.542930 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.542930 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.543247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.543409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.543590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.543754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.543754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.544095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.544213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.544213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.544713 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.545689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.545824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.545960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.545960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.546223 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.546361 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.546361 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.546610 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.546746 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.546746 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.547015 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.547144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.547274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.547405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.547405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.547642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.547789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.547789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.548031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.548176 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.548305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] federation.caching = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.548463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.548463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.548721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.548852 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.548852 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.549096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.549224 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.549224 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.549479 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.549607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.549607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.549841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.549982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.550111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.550236 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.550358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.550487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.550615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.550722 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.550722 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.550917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.551052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.551131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.551252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.551252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.551448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.551525 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.551525 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.551754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.551838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.552001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity.caching = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.552121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.552121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.552364 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.552533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.552533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.552787 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.552913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.552913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.553135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.553259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.553401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.553499 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.553499 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.553705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.553802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.553802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.553986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.554127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.554241 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.554330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.554330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.554525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.554591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.554719 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.554799 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.554799 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.554987 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.555135 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.555267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.555392 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.555392 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.555584 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.555662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] role.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.555662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] role.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.555861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.556003 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.556085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.556204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.556204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.556391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.556529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.556529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.556728 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.556807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.556972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.557099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.557099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.557346 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.557474 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.557474 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.557705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.557849 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.557947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.558080 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.558080 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.558726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.558851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.558973 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.559125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.559231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.559330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.559330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.559536 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.559636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.559754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.559914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.559914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.560153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.560259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.560259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.560508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.560628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.560793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.560916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.560916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.561141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.561408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.561565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.561704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.561827 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.561944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.561944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.562210 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.562352 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.562496 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.562650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.562809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.562962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.562962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.563270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.563431 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.563431 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.563728 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.563894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.564061 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.564218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.564218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.564529 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.564675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.564675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.564993 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.565154 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.565154 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.565449 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.565606 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.565773 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.565966 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.565966 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.566489 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.566657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.566657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.567003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.567151 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.567151 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.567364 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.567444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.567444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.567649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.567794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.connection = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.567939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.568084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.568084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.568328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.568581 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.568581 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.568846 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.568979 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.568979 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.569220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.569353 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.569353 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.569606 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.569734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.569860 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.569987 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.569987 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.570221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.570369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.570369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.570617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.570750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.570881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.571008 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.571008 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.571261 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.571397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.571397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.571627 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.571760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 16:26:42.571907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.571907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.571907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.571907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.571907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.571907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.571907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.572783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.572783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.573020 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.573222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.573540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.573837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.574177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.574478 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.574799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.575108 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.575419 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.575737 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.576060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.576369 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.576739 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.577033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.577356 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.577657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.577965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.578271 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.578588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.578917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.579218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.579526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.580371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] resource.caching = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.580728 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.581043 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.581349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.581659 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.581978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.582282 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.582601 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.582905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.583214 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.583524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] role.caching = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.583841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] role.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.584170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.584541 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.584861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.585177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.585493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.585811 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.586126 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.586442 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.586761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.587060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.587382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.587695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.588016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.588340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.588854 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.589158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.589481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.589800 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.590146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.590463 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.590777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.591095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.591414 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.591720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.592026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.592325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.592660 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.592980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.593312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.593628 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.593934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.594272 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.594581 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.594892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] token.caching = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.595201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.596139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.596509 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.596844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.597156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.597486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.597809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.598167 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.598485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.598794 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.599105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.599410 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.599730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.600046 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.600179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.600467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.600594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.600726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.600875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.601200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.601576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.connection = **** {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.601962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.602241 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.602587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.602851 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.603163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.603463 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.603819 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.604125 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.604505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.604825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.605155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.605465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.605789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.606102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.606427 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.606733 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.607049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.607366 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:26:42.607633 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 16:26:42.609609 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.641711 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.642005 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.642841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.650794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.651027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.651819 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.671596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.671873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.719162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.719355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.804703 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.804938 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.805094 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:42.805215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.044682 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.044861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.044950 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.045031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.045294 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.045382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.045637 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.045723 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.045978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.046066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.046320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.046409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.088365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.088605 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.088753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.088841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.089103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.089187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.089450 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.089545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.089789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.089874 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.090126 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.090210 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.090463 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.090549 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.104730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.111656 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.117464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.117680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.117905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.118127 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.118343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.118558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.118771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.122987 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.123311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.123542 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.123769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.123991 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.124212 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.124452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:26:43.136063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.136267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 16:26:43.136432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.136637 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.136742 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 16:26:43.136823 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 16:26:43.137071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.137155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.137315 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 16:26:43.137414 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.137596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.137678 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.137854 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.137937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.138447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.138726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.148355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.148659 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.148932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.149152 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.149253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.149440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.149754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.150046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.156159 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.157243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.159614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.159897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.160180 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.160490 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.160778 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.160876 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.161043 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.161127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.161319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.161456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.161592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.161677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.161865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.161947 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.162147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.162237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.162417 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.162499 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.162696 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.163086 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.163373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.163652 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.185788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.187251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.187338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.187536 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.188869 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.189190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.192252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.192557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.192843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.194090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.194380 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.194676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.199194 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.199476 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.201145 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.201440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.203570 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.203853 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.204134 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.204424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.204710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.205069 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.205346 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.205613 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.205697 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.205960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.206252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.206548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.206831 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.207117 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.207396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.207675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.223527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.223815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.226198 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.226494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.227720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.230487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.230977 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.231272 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.233776 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.234090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.235497 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.238970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.239222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.242376 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.242512 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.242792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.243078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.243364 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.243640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.243928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.245724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.246013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.257884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.257884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.257884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.257884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.259854 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.260158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.260448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.260789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.263402 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.263717 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.264075 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.264348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.268418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.270217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.272267 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.272887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.273179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.273469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.273761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.273899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.274018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.274147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.276836 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.277130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.277439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.277738 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.278024 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.286055 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.286341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.286623 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.286904 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.287187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.287469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.287761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.290345 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.290643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.290933 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.291223 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.291515 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.291804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.292085 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.303647 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.306683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.308125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.309620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.311063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.311455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.312568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.312857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.313150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.314462 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.316041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.317588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.317875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.318179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.319710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.321582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.324179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.324506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.324812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.324918 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.325093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.326475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.329622 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.330510 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.330810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.331103 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.333951 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.334260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.334546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.339531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.339846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.340138 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.341981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.342348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.342707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.343054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.343401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.343750 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.344119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.344529 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.344878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.345225 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.346215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.346570 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.346914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.347243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.348140 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:26:43.348467 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.348766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.349054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.349349 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.349682 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.350036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.350248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.350559 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.350961 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.351262 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.351552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.351831 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.352099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:26:43.383429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563e7f0aa9c0 pid: 81844 (default app) Aug 24 16:26:43.387195 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe0c30-6ca4-473c-9025-c42b2cf6b440 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:43.387514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe0c30-6ca4-473c-9025-c42b2cf6b440 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:43.387745 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe0c30-6ca4-473c-9025-c42b2cf6b440 None None] PATH_INFO: `/v3` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:43.388196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563e7f0aa9c0 pid: 81843 (default app) Aug 24 16:26:43.392911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1/1] 10.222.0.70 () {50 vars in 791 bytes} [Wed Aug 24 16:26:43 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:51.589595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba059c69-ab14-479d-b8ed-fe30b39c6651 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.590227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba059c69-ab14-479d-b8ed-fe30b39c6651 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.590422 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba059c69-ab14-479d-b8ed-fe30b39c6651 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:51.596998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1/2] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:26:51 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:51.606380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d40c6-dcf7-4059-98a3-4e9066cef7c6 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.606964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d40c6-dcf7-4059-98a3-4e9066cef7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.607231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d40c6-dcf7-4059-98a3-4e9066cef7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:51.679847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: INFO dbcounter [None req-d14d40c6-dcf7-4059-98a3-4e9066cef7c6 None None] Registered counter for database keystone Aug 24 16:26:51.702411 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG oslo_db.sqlalchemy.engines [None req-d14d40c6-dcf7-4059-98a3-4e9066cef7c6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81844) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 16:26:51.709735 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writer thread running {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 16:26:51.904570 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG passlib.handlers.bcrypt [None req-d14d40c6-dcf7-4059-98a3-4e9066cef7c6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81844) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 16:26:51.905084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG passlib.handlers.bcrypt [None req-d14d40c6-dcf7-4059-98a3-4e9066cef7c6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81844) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 16:26:51.962868 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-d14d40c6-dcf7-4059-98a3-4e9066cef7c6 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:51.964536 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d14d40c6-dcf7-4059-98a3-4e9066cef7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:52.103616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 2/3] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 16:26:52.581318 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba059c69-ab14-479d-b8ed-fe30b39c6651 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:52.650180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: INFO dbcounter [None req-ba059c69-ab14-479d-b8ed-fe30b39c6651 None None] Registered counter for database keystone Aug 24 16:26:52.670562 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG oslo_db.sqlalchemy.engines [None req-ba059c69-ab14-479d-b8ed-fe30b39c6651 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81843) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 16:26:52.680899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writer thread running {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 16:26:52.804445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba059c69-ab14-479d-b8ed-fe30b39c6651 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:52.808883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9baca9f-3ca0-4211-8c7e-a23a87daf8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:52.809531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baca9f-3ca0-4211-8c7e-a23a87daf8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.809818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baca9f-3ca0-4211-8c7e-a23a87daf8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.810091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baca9f-3ca0-4211-8c7e-a23a87daf8b4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.819545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9baca9f-3ca0-4211-8c7e-a23a87daf8b4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.873160 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e9baca9f-3ca0-4211-8c7e-a23a87daf8b4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 16:26:52.873508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9baca9f-3ca0-4211-8c7e-a23a87daf8b4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.884736 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 2/4] 10.222.0.70 () {58 vars in 1324 bytes} [Wed Aug 24 16:26:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:26:52.890849 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d40c6-dcf7-4059-98a3-4e9066cef7c6 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:52.906739 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d40c6-dcf7-4059-98a3-4e9066cef7c6 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:52.912084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a20b2-bee1-4043-ab37-152f5be76497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:52.913286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a20b2-bee1-4043-ab37-152f5be76497 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.913575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a20b2-bee1-4043-ab37-152f5be76497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.913853 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a20b2-bee1-4043-ab37-152f5be76497 admin admin] PATH_INFO: `/v3/projects/8f3b91fa60b4447a866a572209ebc37f` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.915547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a20b2-bee1-4043-ab37-152f5be76497 admin admin] RBAC: Authorizing `identity:get_project(project_id=8f3b91fa60b4447a866a572209ebc37f)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.969365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: WARNING keystone.common.rbac_enforcer.enforcer [None req-168a20b2-bee1-4043-ab37-152f5be76497 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 16:26:52.969710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a20b2-bee1-4043-ab37-152f5be76497 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.975085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 3/5] 10.222.0.70 () {56 vars in 1207 bytes} [Wed Aug 24 16:26:52 2022] GET /identity/v3/projects/8f3b91fa60b4447a866a572209ebc37f => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:54.918883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e566b1-3690-4555-924e-6873e70aa827 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:54.919454 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e566b1-3690-4555-924e-6873e70aa827 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:54.919454 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e566b1-3690-4555-924e-6873e70aa827 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:54.923685 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 3/6] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:26:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:54.930170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90037ed2-4219-4910-a171-62f433509ddc None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:54.930702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90037ed2-4219-4910-a171-62f433509ddc None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:54.930787 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90037ed2-4219-4910-a171-62f433509ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:54.952058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-90037ed2-4219-4910-a171-62f433509ddc None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:54.953703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-90037ed2-4219-4910-a171-62f433509ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:54.962547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 4/7] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 16:26:55.231748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e566b1-3690-4555-924e-6873e70aa827 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:55.244897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e566b1-3690-4555-924e-6873e70aa827 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:55.249670 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fdacd-6585-4b92-9420-c96257e4fc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:55.250301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fdacd-6585-4b92-9420-c96257e4fc65 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.250588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fdacd-6585-4b92-9420-c96257e4fc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:55.250869 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fdacd-6585-4b92-9420-c96257e4fc65 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:55.260355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fdacd-6585-4b92-9420-c96257e4fc65 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:55.262200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fdacd-6585-4b92-9420-c96257e4fc65 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:55.272807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 4/8] 10.222.0.70 () {58 vars in 1324 bytes} [Wed Aug 24 16:26:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 16:26:55.278907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90037ed2-4219-4910-a171-62f433509ddc None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:55.288700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90037ed2-4219-4910-a171-62f433509ddc None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:55.293235 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89c436-4046-4c5a-9562-ad329c23a265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:55.293893 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89c436-4046-4c5a-9562-ad329c23a265 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.294176 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89c436-4046-4c5a-9562-ad329c23a265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:55.294452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89c436-4046-4c5a-9562-ad329c23a265 admin admin] PATH_INFO: `/v3/users/cffc436199314734957ccd7232ed1e29` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:55.295828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89c436-4046-4c5a-9562-ad329c23a265 admin admin] RBAC: Authorizing `identity:get_user(user_id=cffc436199314734957ccd7232ed1e29)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:55.297734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89c436-4046-4c5a-9562-ad329c23a265 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:55.299934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 5/9] 10.222.0.70 () {56 vars in 1198 bytes} [Wed Aug 24 16:26:55 2022] GET /identity/v3/users/cffc436199314734957ccd7232ed1e29 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:57.368150 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ca75f-6762-4b45-b504-4685dbc50795 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.368826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ca75f-6762-4b45-b504-4685dbc50795 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.369167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ca75f-6762-4b45-b504-4685dbc50795 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.371782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 5/10] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:26:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:57.380680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1dc92-eccf-4c0e-a1be-c3e8946e9e30 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.381218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1dc92-eccf-4c0e-a1be-c3e8946e9e30 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.381312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1dc92-eccf-4c0e-a1be-c3e8946e9e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.401398 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-cde1dc92-eccf-4c0e-a1be-c3e8946e9e30 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:57.402797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-cde1dc92-eccf-4c0e-a1be-c3e8946e9e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:57.403858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b460eef-ba10-4eb3-b2fb-fcd4316b433b None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.404243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b460eef-ba10-4eb3-b2fb-fcd4316b433b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.404586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b460eef-ba10-4eb3-b2fb-fcd4316b433b None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.406087 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 6/11] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:57.412636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e55e6-d682-49f4-914e-f884de59ffd9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.412943 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e55e6-d682-49f4-914e-f884de59ffd9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.413179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e55e6-d682-49f4-914e-f884de59ffd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.457793 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG passlib.handlers.bcrypt [None req-e66e55e6-d682-49f4-914e-f884de59ffd9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81843) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 16:26:57.458297 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG passlib.handlers.bcrypt [None req-e66e55e6-d682-49f4-914e-f884de59ffd9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81843) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 16:26:57.504631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e66e55e6-d682-49f4-914e-f884de59ffd9 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:57.506253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e66e55e6-d682-49f4-914e-f884de59ffd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:57.522416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 6/12] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:57.617595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 7/13] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:57.790831 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1dc92-eccf-4c0e-a1be-c3e8946e9e30 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.818919 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:57.888647 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e55e6-d682-49f4-914e-f884de59ffd9 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.913908 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1dc92-eccf-4c0e-a1be-c3e8946e9e30 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.949992 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a58fb-eba8-4844-9b39-566d62427513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.950634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a58fb-eba8-4844-9b39-566d62427513 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.950915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a58fb-eba8-4844-9b39-566d62427513 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.951188 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a58fb-eba8-4844-9b39-566d62427513 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.964257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a58fb-eba8-4844-9b39-566d62427513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-193a58fb-eba8-4844-9b39-566d62427513 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:57.973000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:57.974048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:57.974947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:57.974947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:57.974947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:57.974947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:57.974947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:57.974947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:57.974947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:57.974947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:57.974947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:57.974947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:57.974947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:57.974947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:26:57.974947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 7/14] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:57.980264 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a58fb-eba8-4844-9b39-566d62427513 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:58.035526 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e55e6-d682-49f4-914e-f884de59ffd9 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.038193 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:58.076781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e6054-3616-4c93-999d-1c3c5cadd662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.077550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e6054-3616-4c93-999d-1c3c5cadd662 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.077835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e6054-3616-4c93-999d-1c3c5cadd662 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.078113 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e6054-3616-4c93-999d-1c3c5cadd662 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.089766 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:58.109745 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a58fb-eba8-4844-9b39-566d62427513 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.161783 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e6054-3616-4c93-999d-1c3c5cadd662 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.162165 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116d524-8e9a-4b7b-909f-0a705709fe26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.163220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116d524-8e9a-4b7b-909f-0a705709fe26 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.163724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116d524-8e9a-4b7b-909f-0a705709fe26 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.163883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e6054-3616-4c93-999d-1c3c5cadd662 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.164209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116d524-8e9a-4b7b-909f-0a705709fe26 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.165472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116d524-8e9a-4b7b-909f-0a705709fe26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:58.166011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116d524-8e9a-4b7b-909f-0a705709fe26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.168681 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116d524-8e9a-4b7b-909f-0a705709fe26 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.193242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 8/15] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:58.200084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116d524-8e9a-4b7b-909f-0a705709fe26 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:58.292095 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 8/16] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:26:58.297754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e6054-3616-4c93-999d-1c3c5cadd662 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:58.301586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:58.337491 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116d524-8e9a-4b7b-909f-0a705709fe26 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.389867 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f18f4-7d29-4c42-a366-0785544d9020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.390952 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f18f4-7d29-4c42-a366-0785544d9020 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.391435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f18f4-7d29-4c42-a366-0785544d9020 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.392506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f18f4-7d29-4c42-a366-0785544d9020 None admin] PATH_INFO: `/v3/users/cffc436199314734957ccd7232ed1e29` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.394622 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f18f4-7d29-4c42-a366-0785544d9020 None admin] RBAC: Authorizing `identity:get_user(user_id=cffc436199314734957ccd7232ed1e29)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.397442 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f18f4-7d29-4c42-a366-0785544d9020 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.400331 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 9/17] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:58 2022] GET /identity/v3/users/cffc436199314734957ccd7232ed1e29 => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:58.409571 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f18f4-7d29-4c42-a366-0785544d9020 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:58.416281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e6054-3616-4c93-999d-1c3c5cadd662 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.437085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:58.459877 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3bcc7-60c4-488d-a987-9800d36318d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.460847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3bcc7-60c4-488d-a987-9800d36318d2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.461127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3bcc7-60c4-488d-a987-9800d36318d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.461398 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3bcc7-60c4-488d-a987-9800d36318d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.482170 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3bcc7-60c4-488d-a987-9800d36318d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.483869 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3bcc7-60c4-488d-a987-9800d36318d2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-2fb3bcc7-60c4-488d-a987-9800d36318d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:58.504356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:58.511153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:58.511992 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:58.511992 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:58.511992 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:26:58.511992 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 9/18] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:58.512207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3bcc7-60c4-488d-a987-9800d36318d2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:58.545206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f18f4-7d29-4c42-a366-0785544d9020 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.585238 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:58.588389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ba148-804d-44d4-9345-120aa0715ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.589061 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ba148-804d-44d4-9345-120aa0715ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.589342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ba148-804d-44d4-9345-120aa0715ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.589629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ba148-804d-44d4-9345-120aa0715ce2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.591535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ba148-804d-44d4-9345-120aa0715ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.593325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ba148-804d-44d4-9345-120aa0715ce2 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.609468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 10/19] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:58.609468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ba148-804d-44d4-9345-120aa0715ce2 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:58.662134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3bcc7-60c4-488d-a987-9800d36318d2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.712888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a8aee-b500-4308-8ede-a5a8999ceb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.713540 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a8aee-b500-4308-8ede-a5a8999ceb34 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.713829 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a8aee-b500-4308-8ede-a5a8999ceb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.714108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a8aee-b500-4308-8ede-a5a8999ceb34 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.714814 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a8aee-b500-4308-8ede-a5a8999ceb34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:58.715124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a8aee-b500-4308-8ede-a5a8999ceb34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.716617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a8aee-b500-4308-8ede-a5a8999ceb34 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.741255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 10/20] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:58.748705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:58.774467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ba148-804d-44d4-9345-120aa0715ce2 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.825141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af9aa2-6c71-4f4f-9857-640f159c2018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.825925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af9aa2-6c71-4f4f-9857-640f159c2018 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.826275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af9aa2-6c71-4f4f-9857-640f159c2018 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.826610 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af9aa2-6c71-4f4f-9857-640f159c2018 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.827499 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af9aa2-6c71-4f4f-9857-640f159c2018 None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.domain.id=default, user.id=cffc436199314734957ccd7232ed1e29) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:58.827873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af9aa2-6c71-4f4f-9857-640f159c2018 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.829718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af9aa2-6c71-4f4f-9857-640f159c2018 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.838797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 11/21] 10.222.0.70 () {56 vars in 1322 bytes} [Wed Aug 24 16:26:58 2022] GET /identity/v3/role_assignments?user.id=cffc436199314734957ccd7232ed1e29&scope.domain.id=default&role.id=e7f48343fc1542979ab4459865109e30 => generated 233 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:01.122580 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1efb38-5e9a-4c65-bb2a-9f17ab72b486 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:01.123359 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1efb38-5e9a-4c65-bb2a-9f17ab72b486 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:01.123359 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1efb38-5e9a-4c65-bb2a-9f17ab72b486 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:01.124548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 11/22] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:01.131439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f981c-08ba-403a-a1f5-8f2280286c8b None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:01.132186 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f981c-08ba-403a-a1f5-8f2280286c8b None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:01.132186 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f981c-08ba-403a-a1f5-8f2280286c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:01.158800 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-659f981c-08ba-403a-a1f5-8f2280286c8b None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:01.160954 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-659f981c-08ba-403a-a1f5-8f2280286c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:01.219974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5c123-e3fd-4f75-b339-b7330412b39b None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:01.221462 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5c123-e3fd-4f75-b339-b7330412b39b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:01.222479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5c123-e3fd-4f75-b339-b7330412b39b None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:01.227266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 12/23] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:01.234454 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861e1a0-3b56-4195-98a0-9919af60165c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:01.236172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861e1a0-3b56-4195-98a0-9919af60165c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:01.237690 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861e1a0-3b56-4195-98a0-9919af60165c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:01.282840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 12/24] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:01.285467 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3861e1a0-3b56-4195-98a0-9919af60165c None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:01.286696 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3861e1a0-3b56-4195-98a0-9919af60165c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:01.400727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 13/25] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:01.690672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f981c-08ba-403a-a1f5-8f2280286c8b None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:01.792453 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:01.830017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f981c-08ba-403a-a1f5-8f2280286c8b None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:01.885836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810f12e-c7c4-46f7-a61b-2b8646030278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:01.886031 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c033d-56d5-4e73-9f89-20ef54ac8ee7 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:01.886388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c033d-56d5-4e73-9f89-20ef54ac8ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:01.886515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810f12e-c7c4-46f7-a61b-2b8646030278 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:01.886762 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810f12e-c7c4-46f7-a61b-2b8646030278 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:01.886860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c033d-56d5-4e73-9f89-20ef54ac8ee7 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:01.887592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810f12e-c7c4-46f7-a61b-2b8646030278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:01.897012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 14/26] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:01 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:01.903601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6edaee-3a72-47b7-bdf9-5250fc1ef322 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:01.904591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6edaee-3a72-47b7-bdf9-5250fc1ef322 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:01.905383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6edaee-3a72-47b7-bdf9-5250fc1ef322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:01.905649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0810f12e-c7c4-46f7-a61b-2b8646030278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:01.907199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0810f12e-c7c4-46f7-a61b-2b8646030278 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-0810f12e-c7c4-46f7-a61b-2b8646030278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:01.942999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:01.944696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:01.946273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:01.946273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:01.946273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:01.946273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 13/27] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:01.950019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8c6edaee-3a72-47b7-bdf9-5250fc1ef322 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:01.950472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810f12e-c7c4-46f7-a61b-2b8646030278 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:01.951267 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8c6edaee-3a72-47b7-bdf9-5250fc1ef322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:02.057314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:02.143171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 15/28] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:02.182595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6edaee-3a72-47b7-bdf9-5250fc1ef322 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:02.294933 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810f12e-c7c4-46f7-a61b-2b8646030278 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:02.374227 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:02.394279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a151ac7-d2b6-4432-b887-49ffeb4d5ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:02.394960 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a151ac7-d2b6-4432-b887-49ffeb4d5ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:02.395376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a151ac7-d2b6-4432-b887-49ffeb4d5ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:02.395640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a151ac7-d2b6-4432-b887-49ffeb4d5ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.396329 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a151ac7-d2b6-4432-b887-49ffeb4d5ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:02.396680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a151ac7-d2b6-4432-b887-49ffeb4d5ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:02.398130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a151ac7-d2b6-4432-b887-49ffeb4d5ce3 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:02.419103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 14/29] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:02.432417 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a151ac7-d2b6-4432-b887-49ffeb4d5ce3 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:02.496531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6edaee-3a72-47b7-bdf9-5250fc1ef322 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:02.557518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a151ac7-d2b6-4432-b887-49ffeb4d5ce3 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:02.602372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cc2ee-4917-48da-a48d-131e3d5c6faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:02.603032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cc2ee-4917-48da-a48d-131e3d5c6faf None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:02.603318 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cc2ee-4917-48da-a48d-131e3d5c6faf None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:02.603594 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cc2ee-4917-48da-a48d-131e3d5c6faf None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.604287 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cc2ee-4917-48da-a48d-131e3d5c6faf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:02.605806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cc2ee-4917-48da-a48d-131e3d5c6faf None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:02.617001 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cdb75c-2734-42f1-88ce-6a8e2a0b2c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:02.617666 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdb75c-2734-42f1-88ce-6a8e2a0b2c30 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:02.617950 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdb75c-2734-42f1-88ce-6a8e2a0b2c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:02.618293 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdb75c-2734-42f1-88ce-6a8e2a0b2c30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.637498 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cdb75c-2734-42f1-88ce-6a8e2a0b2c30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-78cdb75c-2734-42f1-88ce-6a8e2a0b2c30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:02.646421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:02.647383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:02.648196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:02.648196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:02.648196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:02.648196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:02.648196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:02.648196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:02.648196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:02.648196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:02.648196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:02.648196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:02.648196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:02.648196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:02.648196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 16/30] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 466 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:02.649525 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cdb75c-2734-42f1-88ce-6a8e2a0b2c30 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:02.690145 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:02.781675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cdb75c-2734-42f1-88ce-6a8e2a0b2c30 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:02.856297 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0bce9-bea0-4574-9dc0-318d00db7bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:02.857382 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0bce9-bea0-4574-9dc0-318d00db7bd0 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:02.857870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0bce9-bea0-4574-9dc0-318d00db7bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:02.858670 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0bce9-bea0-4574-9dc0-318d00db7bd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.859885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0bce9-bea0-4574-9dc0-318d00db7bd0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:02.863074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0bce9-bea0-4574-9dc0-318d00db7bd0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:02.893784 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 15/31] 10.222.0.70 () {60 vars in 1141 bytes} [Wed Aug 24 16:27:02 2022] POST /identity/v3/projects => generated 301 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:27:02.895471 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a88c0a-b00b-4858-87c2-d38fdcbedddf None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:02.895751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a88c0a-b00b-4858-87c2-d38fdcbedddf None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:02.896042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a88c0a-b00b-4858-87c2-d38fdcbedddf None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.897685 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 16/32] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:02.898680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a88c0a-b00b-4858-87c2-d38fdcbedddf None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:02.957467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:03.046347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a88c0a-b00b-4858-87c2-d38fdcbedddf None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:03.168375 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b695b97-7ed9-4bee-9671-438c20f03743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:03.169476 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b695b97-7ed9-4bee-9671-438c20f03743 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.169989 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b695b97-7ed9-4bee-9671-438c20f03743 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.170489 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b695b97-7ed9-4bee-9671-438c20f03743 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.171412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b695b97-7ed9-4bee-9671-438c20f03743 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:03.171971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b695b97-7ed9-4bee-9671-438c20f03743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:03.174496 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b695b97-7ed9-4bee-9671-438c20f03743 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:03.175885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 17/33] 10.222.0.70 () {60 vars in 1131 bytes} [Wed Aug 24 16:27:02 2022] POST /identity/v3/roles => generated 225 bytes in 527 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:03.176613 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824691a-6bd0-4ffd-9c01-61f2fc83e135 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.176912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824691a-6bd0-4ffd-9c01-61f2fc83e135 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.177254 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824691a-6bd0-4ffd-9c01-61f2fc83e135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.207268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 17/34] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:03.214918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b695b97-7ed9-4bee-9671-438c20f03743 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:03.215199 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e824691a-6bd0-4ffd-9c01-61f2fc83e135 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:03.217210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e824691a-6bd0-4ffd-9c01-61f2fc83e135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:03.390906 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 18/35] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:03.392619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38753d-5282-457d-87b5-00a8f0dbb07a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.392913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38753d-5282-457d-87b5-00a8f0dbb07a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.393196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38753d-5282-457d-87b5-00a8f0dbb07a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.394930 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 19/36] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:03.396166 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887c6ab-13ed-4db7-b203-7b2b8bbbcd60 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.400182 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887c6ab-13ed-4db7-b203-7b2b8bbbcd60 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.400753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887c6ab-13ed-4db7-b203-7b2b8bbbcd60 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.402922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 20/37] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:03.404591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc2fb4-1a13-49f7-9091-3412402954a8 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.405070 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc2fb4-1a13-49f7-9091-3412402954a8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.405547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc2fb4-1a13-49f7-9091-3412402954a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.440185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-debc2fb4-1a13-49f7-9091-3412402954a8 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:03.442189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-debc2fb4-1a13-49f7-9091-3412402954a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:03.454863 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:03.459266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b695b97-7ed9-4bee-9671-438c20f03743 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:03.487987 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:03.510189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c3556-2ac0-417d-8a44-ec5b380cd713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:03.511606 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c3556-2ac0-417d-8a44-ec5b380cd713 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.512089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c3556-2ac0-417d-8a44-ec5b380cd713 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.512644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c3556-2ac0-417d-8a44-ec5b380cd713 None admin] PATH_INFO: `/v3/users/cffc436199314734957ccd7232ed1e29` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.516072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c3556-2ac0-417d-8a44-ec5b380cd713 None admin] RBAC: Authorizing `identity:get_user(user_id=cffc436199314734957ccd7232ed1e29)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:03.518588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c3556-2ac0-417d-8a44-ec5b380cd713 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:03.524497 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 18/38] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:03 2022] GET /identity/v3/users/cffc436199314734957ccd7232ed1e29 => generated 247 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:03.526653 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bebd4-6cd9-431e-be86-ba783594d296 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.527000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bebd4-6cd9-431e-be86-ba783594d296 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.527279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bebd4-6cd9-431e-be86-ba783594d296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.548716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-d21bebd4-6cd9-431e-be86-ba783594d296 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:03.550321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d21bebd4-6cd9-431e-be86-ba783594d296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:03.576706 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 21/39] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:03.577978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc2fb4-1a13-49f7-9091-3412402954a8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:03.678861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 19/40] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:03.680220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bebd4-6cd9-431e-be86-ba783594d296 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:03.783973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc2fb4-1a13-49f7-9091-3412402954a8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:03.828065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caae44d-9b1b-4292-bcfd-66be2783165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:03.829691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caae44d-9b1b-4292-bcfd-66be2783165f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.830197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caae44d-9b1b-4292-bcfd-66be2783165f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.830703 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caae44d-9b1b-4292-bcfd-66be2783165f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.832649 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caae44d-9b1b-4292-bcfd-66be2783165f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:03.834663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caae44d-9b1b-4292-bcfd-66be2783165f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:03.841846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 22/41] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:03.843171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caae44d-9b1b-4292-bcfd-66be2783165f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:03.875038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:03.977836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:03.981723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caae44d-9b1b-4292-bcfd-66be2783165f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:03.983064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bebd4-6cd9-431e-be86-ba783594d296 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:04.036185 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d336a89-a0de-41fb-a8f1-b4b1b1bf295c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:04.036839 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d336a89-a0de-41fb-a8f1-b4b1b1bf295c None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:04.037130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d336a89-a0de-41fb-a8f1-b4b1b1bf295c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:04.037375 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb7e0e-4b32-4a7f-8b13-98e871649d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:04.037545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d336a89-a0de-41fb-a8f1-b4b1b1bf295c None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:04.038014 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb7e0e-4b32-4a7f-8b13-98e871649d6b None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:04.038179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d336a89-a0de-41fb-a8f1-b4b1b1bf295c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:04.038310 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb7e0e-4b32-4a7f-8b13-98e871649d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:04.038563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb7e0e-4b32-4a7f-8b13-98e871649d6b None admin] PATH_INFO: `/v3/domains/default/users/cffc436199314734957ccd7232ed1e29/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:04.039604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d336a89-a0de-41fb-a8f1-b4b1b1bf295c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:04.041127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb7e0e-4b32-4a7f-8b13-98e871649d6b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cffc436199314734957ccd7232ed1e29, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:04.042645 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb7e0e-4b32-4a7f-8b13-98e871649d6b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:04.130363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 20/42] 10.222.0.70 () {60 vars in 1131 bytes} [Wed Aug 24 16:27:03 2022] POST /identity/v3/roles => generated 219 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:04.131233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d336a89-a0de-41fb-a8f1-b4b1b1bf295c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:04.173514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 23/43] 10.222.0.70 () {58 vars in 1361 bytes} [Wed Aug 24 16:27:03 2022] PUT /identity/v3/domains/default/users/cffc436199314734957ccd7232ed1e29/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:27:04.175097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb7e0e-4b32-4a7f-8b13-98e871649d6b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:04.318968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d336a89-a0de-41fb-a8f1-b4b1b1bf295c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:04.378688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76903dcd-fcad-4c2b-98c3-6e25e4adde62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:04.379788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76903dcd-fcad-4c2b-98c3-6e25e4adde62 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:04.380314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76903dcd-fcad-4c2b-98c3-6e25e4adde62 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:04.380847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76903dcd-fcad-4c2b-98c3-6e25e4adde62 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:04.382168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76903dcd-fcad-4c2b-98c3-6e25e4adde62 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:04.384979 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76903dcd-fcad-4c2b-98c3-6e25e4adde62 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:04.397645 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:04.422663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb7e0e-4b32-4a7f-8b13-98e871649d6b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:04.474223 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 21/44] 10.222.0.70 () {60 vars in 1131 bytes} [Wed Aug 24 16:27:04 2022] POST /identity/v3/roles => generated 223 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:04.477543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df597e6c-0ca7-47ec-a0b2-9c6cab383b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:04.478400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df597e6c-0ca7-47ec-a0b2-9c6cab383b14 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:04.478931 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df597e6c-0ca7-47ec-a0b2-9c6cab383b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:04.479394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df597e6c-0ca7-47ec-a0b2-9c6cab383b14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:04.481090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df597e6c-0ca7-47ec-a0b2-9c6cab383b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:04.486811 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df597e6c-0ca7-47ec-a0b2-9c6cab383b14 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:04.486811 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 24/45] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:04.491001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76903dcd-fcad-4c2b-98c3-6e25e4adde62 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:04.518138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:04.656537 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76903dcd-fcad-4c2b-98c3-6e25e4adde62 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:04.712998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150b42d-ac47-49c6-b8f5-000329c736a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:04.713749 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150b42d-ac47-49c6-b8f5-000329c736a8 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:04.714041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150b42d-ac47-49c6-b8f5-000329c736a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:04.714326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150b42d-ac47-49c6-b8f5-000329c736a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:04.715032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150b42d-ac47-49c6-b8f5-000329c736a8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:04.716718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150b42d-ac47-49c6-b8f5-000329c736a8 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:04.855166 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 22/46] 10.222.0.70 () {60 vars in 1141 bytes} [Wed Aug 24 16:27:04 2022] POST /identity/v3/projects => generated 312 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:27:06.250646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e93d78-8004-4337-9bd3-6f99aa6bdba0 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:06.251685 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e93d78-8004-4337-9bd3-6f99aa6bdba0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:06.251988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e93d78-8004-4337-9bd3-6f99aa6bdba0 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:06.253810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 25/47] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:06.259427 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78136cf1-cdeb-4674-84bf-7436a8200076 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:06.259853 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78136cf1-cdeb-4674-84bf-7436a8200076 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:06.260025 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78136cf1-cdeb-4674-84bf-7436a8200076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:06.279519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-78136cf1-cdeb-4674-84bf-7436a8200076 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:06.280757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-78136cf1-cdeb-4674-84bf-7436a8200076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:06.345657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:06.350403 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 23/48] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:06.619523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e93d78-8004-4337-9bd3-6f99aa6bdba0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:06.730418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e93d78-8004-4337-9bd3-6f99aa6bdba0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:06.770072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0124a-ebf4-4503-b976-834745ca588c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:06.770373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0124a-ebf4-4503-b976-834745ca588c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:06.770777 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0124a-ebf4-4503-b976-834745ca588c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:06.770931 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0124a-ebf4-4503-b976-834745ca588c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:06.781424 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0124a-ebf4-4503-b976-834745ca588c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-9bc0124a-ebf4-4503-b976-834745ca588c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:06.786325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:06.787205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:06.787993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:06.787993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:06.787993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:06.787993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:06.787993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:06.787993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:06.787993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:06.787993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:06.787993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:06.787993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:06.787993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:06.787993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:06.787993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 26/49] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:06.793446 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78136cf1-cdeb-4674-84bf-7436a8200076 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:06.819213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afcff37-ee95-457d-ab3c-a12e66e8d172 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:06.819511 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afcff37-ee95-457d-ab3c-a12e66e8d172 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:06.819786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afcff37-ee95-457d-ab3c-a12e66e8d172 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:06.821178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 27/50] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:06.826177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef14b4d-d97a-4418-ab0d-a4d365117d3c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:06.826445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef14b4d-d97a-4418-ab0d-a4d365117d3c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:06.826725 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef14b4d-d97a-4418-ab0d-a4d365117d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:06.848527 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5ef14b4d-d97a-4418-ab0d-a4d365117d3c None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:06.849758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5ef14b4d-d97a-4418-ab0d-a4d365117d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:06.857754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:06.899450 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78136cf1-cdeb-4674-84bf-7436a8200076 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:06.920036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 28/51] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:06.935943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938e519-7196-4504-8dea-dbc3a1b93aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:06.936590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938e519-7196-4504-8dea-dbc3a1b93aaf None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:06.936948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938e519-7196-4504-8dea-dbc3a1b93aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:06.937235 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938e519-7196-4504-8dea-dbc3a1b93aaf None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:06.937908 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938e519-7196-4504-8dea-dbc3a1b93aaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:06.938217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938e519-7196-4504-8dea-dbc3a1b93aaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:06.939636 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938e519-7196-4504-8dea-dbc3a1b93aaf None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:06.947964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 24/52] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:06.954356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef14b4d-d97a-4418-ab0d-a4d365117d3c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:07.059226 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef14b4d-d97a-4418-ab0d-a4d365117d3c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:07.096988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb66f8-a8cc-4e5f-a8ed-9b1c71fd3ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:07.097642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb66f8-a8cc-4e5f-a8ed-9b1c71fd3ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:07.097928 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb66f8-a8cc-4e5f-a8ed-9b1c71fd3ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:07.098207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb66f8-a8cc-4e5f-a8ed-9b1c71fd3ee3 None admin] PATH_INFO: `/v3/users/cffc436199314734957ccd7232ed1e29` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:07.099405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb66f8-a8cc-4e5f-a8ed-9b1c71fd3ee3 None admin] RBAC: Authorizing `identity:get_user(user_id=cffc436199314734957ccd7232ed1e29)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:07.100927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb66f8-a8cc-4e5f-a8ed-9b1c71fd3ee3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:07.102715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 29/53] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:06 2022] GET /identity/v3/users/cffc436199314734957ccd7232ed1e29 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:07.108863 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938e519-7196-4504-8dea-dbc3a1b93aaf None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:07.190563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb66f8-a8cc-4e5f-a8ed-9b1c71fd3ee3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:07.203674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:07.216677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:07.261714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938e519-7196-4504-8dea-dbc3a1b93aaf None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:07.299825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f37c52-a544-4e70-bd58-efa7840c0aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:07.300492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f37c52-a544-4e70-bd58-efa7840c0aab None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:07.300783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f37c52-a544-4e70-bd58-efa7840c0aab None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:07.301062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f37c52-a544-4e70-bd58-efa7840c0aab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:07.302186 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f37c52-a544-4e70-bd58-efa7840c0aab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:07.303675 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f37c52-a544-4e70-bd58-efa7840c0aab None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:07.305297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 25/54] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:07.311190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f37c52-a544-4e70-bd58-efa7840c0aab None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:07.316522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb66f8-a8cc-4e5f-a8ed-9b1c71fd3ee3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:07.352343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644b3d9-445b-463c-8eb0-1776a4bce923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:07.353021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644b3d9-445b-463c-8eb0-1776a4bce923 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:07.353318 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644b3d9-445b-463c-8eb0-1776a4bce923 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:07.353595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644b3d9-445b-463c-8eb0-1776a4bce923 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:07.354740 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1644b3d9-445b-463c-8eb0-1776a4bce923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:07.356242 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1644b3d9-445b-463c-8eb0-1776a4bce923 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:07.357926 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 30/55] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:07.363570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644b3d9-445b-463c-8eb0-1776a4bce923 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:07.419299 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f37c52-a544-4e70-bd58-efa7840c0aab None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:07.447217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:07.451227 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:07.460486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbf0c2-c997-4861-8fb3-bcc35c667d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:07.461494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbf0c2-c997-4861-8fb3-bcc35c667d95 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:07.462009 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbf0c2-c997-4861-8fb3-bcc35c667d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:07.462526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbf0c2-c997-4861-8fb3-bcc35c667d95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:07.463835 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbf0c2-c997-4861-8fb3-bcc35c667d95 None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.domain.id=default, user.id=cffc436199314734957ccd7232ed1e29) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:07.464517 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbf0c2-c997-4861-8fb3-bcc35c667d95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:07.467282 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbf0c2-c997-4861-8fb3-bcc35c667d95 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:07.474317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644b3d9-445b-463c-8eb0-1776a4bce923 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:07.478759 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 26/56] 10.222.0.70 () {56 vars in 1322 bytes} [Wed Aug 24 16:27:07 2022] GET /identity/v3/role_assignments?user.id=cffc436199314734957ccd7232ed1e29&scope.domain.id=default&role.id=e7f48343fc1542979ab4459865109e30 => generated 530 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:07.635073 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41297cd1-5db6-4a38-9fb2-a87a653a5270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:07.635712 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41297cd1-5db6-4a38-9fb2-a87a653a5270 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:07.635999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41297cd1-5db6-4a38-9fb2-a87a653a5270 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:07.638952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41297cd1-5db6-4a38-9fb2-a87a653a5270 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:07.638952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41297cd1-5db6-4a38-9fb2-a87a653a5270 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:07.638952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41297cd1-5db6-4a38-9fb2-a87a653a5270 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:07.770525 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 31/57] 10.222.0.70 () {60 vars in 1140 bytes} [Wed Aug 24 16:27:07 2022] POST /identity/v3/projects => generated 298 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:27:09.710399 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3132da-3a87-430a-8d53-d190029c6ee1 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.710921 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3132da-3a87-430a-8d53-d190029c6ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.711112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3132da-3a87-430a-8d53-d190029c6ee1 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.712676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 27/58] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:09.719933 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cfe44-00ff-4e99-908f-34c5e313ef97 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.720504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cfe44-00ff-4e99-908f-34c5e313ef97 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.720990 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cfe44-00ff-4e99-908f-34c5e313ef97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.744877 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b04cfe44-00ff-4e99-908f-34c5e313ef97 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:09.746132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b04cfe44-00ff-4e99-908f-34c5e313ef97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:09.811842 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 32/59] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:10.084430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3132da-3a87-430a-8d53-d190029c6ee1 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:10.194549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3132da-3a87-430a-8d53-d190029c6ee1 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:10.228410 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263f7ca-8bdb-42b0-b4cd-2fa9064ebba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:10.229055 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263f7ca-8bdb-42b0-b4cd-2fa9064ebba9 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:10.229337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263f7ca-8bdb-42b0-b4cd-2fa9064ebba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:10.229607 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263f7ca-8bdb-42b0-b4cd-2fa9064ebba9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:10.230714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263f7ca-8bdb-42b0-b4cd-2fa9064ebba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:10.232147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263f7ca-8bdb-42b0-b4cd-2fa9064ebba9 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:10.233727 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 28/60] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:10.239924 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cfe44-00ff-4e99-908f-34c5e313ef97 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:10.300726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:10.351756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cfe44-00ff-4e99-908f-34c5e313ef97 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:10.387527 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792a9b7-004f-48c3-8953-1803550144f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:10.388141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792a9b7-004f-48c3-8953-1803550144f5 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:10.388448 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792a9b7-004f-48c3-8953-1803550144f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:10.388728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792a9b7-004f-48c3-8953-1803550144f5 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:10.389391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792a9b7-004f-48c3-8953-1803550144f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:10.390851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792a9b7-004f-48c3-8953-1803550144f5 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:10.491984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 33/61] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:27:10 2022] POST /identity/v3/users => generated 275 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:13.583560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27bda8-a17e-4223-b79f-5d02cce64a50 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:13.584545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27bda8-a17e-4223-b79f-5d02cce64a50 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:13.585020 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27bda8-a17e-4223-b79f-5d02cce64a50 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:13.586676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 29/62] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:13.592339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf584d51-1c9d-4164-b8e8-24581e55c970 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:13.592820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf584d51-1c9d-4164-b8e8-24581e55c970 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:13.592996 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf584d51-1c9d-4164-b8e8-24581e55c970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:13.617179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-cf584d51-1c9d-4164-b8e8-24581e55c970 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:13.618775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-cf584d51-1c9d-4164-b8e8-24581e55c970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:13.719065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 34/63] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:13.748887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d4023-95f5-4a2d-85fe-e366b45f0f71 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:13.749537 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d4023-95f5-4a2d-85fe-e366b45f0f71 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:13.750009 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d4023-95f5-4a2d-85fe-e366b45f0f71 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:13.752092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 30/64] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:13.757796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9173b20-b353-4f8e-9245-c56436370358 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:13.758095 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9173b20-b353-4f8e-9245-c56436370358 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:13.758376 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9173b20-b353-4f8e-9245-c56436370358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:13.781186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-d9173b20-b353-4f8e-9245-c56436370358 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:13.782437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d9173b20-b353-4f8e-9245-c56436370358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:13.845402 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:13.877220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 35/65] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:13.990814 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a00f9f-b745-4197-b6a0-c029ba27e188 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:13.991754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a00f9f-b745-4197-b6a0-c029ba27e188 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:13.992503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a00f9f-b745-4197-b6a0-c029ba27e188 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:13.995049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 31/66] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:14.001512 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdd35b-6f44-4685-86af-6fde1f02e1de None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.001902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdd35b-6f44-4685-86af-6fde1f02e1de None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.002135 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdd35b-6f44-4685-86af-6fde1f02e1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.030458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-23cdd35b-6f44-4685-86af-6fde1f02e1de None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:14.031683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-23cdd35b-6f44-4685-86af-6fde1f02e1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:14.109345 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a00f9f-b745-4197-b6a0-c029ba27e188 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:14.131301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 36/67] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:14.214923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdd35b-6f44-4685-86af-6fde1f02e1de None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:14.246239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:14.323357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a00f9f-b745-4197-b6a0-c029ba27e188 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:14.400638 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32c3e8-4b4f-4eea-81af-24e09b3c00b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:14.401609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32c3e8-4b4f-4eea-81af-24e09b3c00b3 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.402091 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32c3e8-4b4f-4eea-81af-24e09b3c00b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.402574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32c3e8-4b4f-4eea-81af-24e09b3c00b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.412413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdd35b-6f44-4685-86af-6fde1f02e1de None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:14.427245 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32c3e8-4b4f-4eea-81af-24e09b3c00b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:14.429335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-1e32c3e8-4b4f-4eea-81af-24e09b3c00b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:14.436275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:14.437205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:14.437957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:14.437957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:14.437957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:14.437957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:14.437957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:14.437957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:14.437957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:14.437957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:14.437957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:14.437957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:14.437957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:14.437957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:14.441574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 32/68] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:14.442813 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32c3e8-4b4f-4eea-81af-24e09b3c00b3 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:14.476457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba8db5-8d07-4e5a-bfe2-be8625aa974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:14.477050 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba8db5-8d07-4e5a-bfe2-be8625aa974a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.477586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba8db5-8d07-4e5a-bfe2-be8625aa974a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.477586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba8db5-8d07-4e5a-bfe2-be8625aa974a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.490641 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba8db5-8d07-4e5a-bfe2-be8625aa974a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-b3ba8db5-8d07-4e5a-bfe2-be8625aa974a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:14.495508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:14.496386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:14.497169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:14.497169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:14.497169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:14.497169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:14.497169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:14.497169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:14.497169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:14.497169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:14.497169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:14.497169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:14.497169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:14.497169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:14.497169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 37/69] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:14.499785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba8db5-8d07-4e5a-bfe2-be8625aa974a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:14.591389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32c3e8-4b4f-4eea-81af-24e09b3c00b3 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:14.622247 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba8db5-8d07-4e5a-bfe2-be8625aa974a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:14.634785 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:14.648228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc437ac9-3588-4cfd-813e-ec909f3387bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:14.648905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc437ac9-3588-4cfd-813e-ec909f3387bb None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.649193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc437ac9-3588-4cfd-813e-ec909f3387bb None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.656666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc437ac9-3588-4cfd-813e-ec909f3387bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.663660 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc437ac9-3588-4cfd-813e-ec909f3387bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:14.665698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5edcd-13f8-4fa5-89d9-6f5309852c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:14.666324 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5edcd-13f8-4fa5-89d9-6f5309852c85 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.666604 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5edcd-13f8-4fa5-89d9-6f5309852c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.666884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5edcd-13f8-4fa5-89d9-6f5309852c85 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.667579 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5edcd-13f8-4fa5-89d9-6f5309852c85 None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:14.667884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5edcd-13f8-4fa5-89d9-6f5309852c85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-cc437ac9-3588-4cfd-813e-ec909f3387bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:14.668661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:14.669725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:14.670950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:14.670950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:14.670950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:14.670950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:14.670950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:14.670950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:14.670950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:14.670950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:14.670950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:14.670950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:14.670950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:27:14.670950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:14.676525 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5edcd-13f8-4fa5-89d9-6f5309852c85 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:14.676661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 33/70] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:14.676661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc840687-0112-47b5-8126-d89098a7a9b5 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.676661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc840687-0112-47b5-8126-d89098a7a9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.676661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc840687-0112-47b5-8126-d89098a7a9b5 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.676661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 34/71] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:14.676661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc840687-0112-47b5-8126-d89098a7a9b5 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:14.689550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 38/72] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:14.690852 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c7d25-48c8-4ff3-a9dd-de1075e3fe2e None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.691140 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c7d25-48c8-4ff3-a9dd-de1075e3fe2e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.691409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c7d25-48c8-4ff3-a9dd-de1075e3fe2e None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.692716 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 39/73] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:14.693439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c7d25-48c8-4ff3-a9dd-de1075e3fe2e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:14.725863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:14.798636 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc840687-0112-47b5-8126-d89098a7a9b5 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:14.818417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c7d25-48c8-4ff3-a9dd-de1075e3fe2e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:14.835307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d831c28-01b3-494d-a1dc-5bcd83120454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:14.835935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d831c28-01b3-494d-a1dc-5bcd83120454 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.836226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d831c28-01b3-494d-a1dc-5bcd83120454 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.836528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d831c28-01b3-494d-a1dc-5bcd83120454 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.837194 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d831c28-01b3-494d-a1dc-5bcd83120454 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:14.837513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d831c28-01b3-494d-a1dc-5bcd83120454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:14.838924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d831c28-01b3-494d-a1dc-5bcd83120454 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:14.848735 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 35/74] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:14.849985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555451e4-473c-4376-8769-9c1adcb6b70f None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.850274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555451e4-473c-4376-8769-9c1adcb6b70f None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.850557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555451e4-473c-4376-8769-9c1adcb6b70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.856213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e763c9-81f2-474b-aace-339e30dfe6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:14.856865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e763c9-81f2-474b-aace-339e30dfe6be None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.857144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e763c9-81f2-474b-aace-339e30dfe6be None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.857416 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e763c9-81f2-474b-aace-339e30dfe6be None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.858164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e763c9-81f2-474b-aace-339e30dfe6be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:14.858504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e763c9-81f2-474b-aace-339e30dfe6be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:14.860014 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e763c9-81f2-474b-aace-339e30dfe6be None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:14.868862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-555451e4-473c-4376-8769-9c1adcb6b70f None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:14.869058 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 40/75] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:14.869748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e763c9-81f2-474b-aace-339e30dfe6be None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:14.870152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-555451e4-473c-4376-8769-9c1adcb6b70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:14.919240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:14.959542 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 36/76] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:14.960857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab523e18-19be-44bd-bd6c-6383aa49ad3d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.961189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab523e18-19be-44bd-bd6c-6383aa49ad3d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.961468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab523e18-19be-44bd-bd6c-6383aa49ad3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.980001 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:14.982326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-ab523e18-19be-44bd-bd6c-6383aa49ad3d None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:14.983696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ab523e18-19be-44bd-bd6c-6383aa49ad3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:14.991075 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e763c9-81f2-474b-aace-339e30dfe6be None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.031150 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828db46-5652-41ba-890a-6658335b3b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.031794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828db46-5652-41ba-890a-6658335b3b90 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.032726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828db46-5652-41ba-890a-6658335b3b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.033023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828db46-5652-41ba-890a-6658335b3b90 None admin] PATH_INFO: `/v3/users/e0ff5ce14df94fc4a1f5f037539e037c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.051553 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828db46-5652-41ba-890a-6658335b3b90 None admin] RBAC: Authorizing `identity:get_user(user_id=e0ff5ce14df94fc4a1f5f037539e037c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.053274 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828db46-5652-41ba-890a-6658335b3b90 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.054992 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 41/77] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:14 2022] GET /identity/v3/users/e0ff5ce14df94fc4a1f5f037539e037c => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:15.055787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828db46-5652-41ba-890a-6658335b3b90 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.064446 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 37/78] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:15.065234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab523e18-19be-44bd-bd6c-6383aa49ad3d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.165078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828db46-5652-41ba-890a-6658335b3b90 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.187893 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab523e18-19be-44bd-bd6c-6383aa49ad3d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.191558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.203310 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc0479-fdec-4d8e-9aef-1b41a799d498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.206568 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc0479-fdec-4d8e-9aef-1b41a799d498 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.206568 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc0479-fdec-4d8e-9aef-1b41a799d498 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.206568 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc0479-fdec-4d8e-9aef-1b41a799d498 None admin] PATH_INFO: `/v3/users/cffc436199314734957ccd7232ed1e29` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.206568 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc0479-fdec-4d8e-9aef-1b41a799d498 None admin] RBAC: Authorizing `identity:get_user(user_id=cffc436199314734957ccd7232ed1e29)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.208015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc0479-fdec-4d8e-9aef-1b41a799d498 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.209716 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 42/79] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/users/cffc436199314734957ccd7232ed1e29 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:15.214784 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc0479-fdec-4d8e-9aef-1b41a799d498 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.243962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.244154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727363bc-e8c4-43d6-8eb6-f65598995f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.245325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727363bc-e8c4-43d6-8eb6-f65598995f75 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.245325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727363bc-e8c4-43d6-8eb6-f65598995f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.245597 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727363bc-e8c4-43d6-8eb6-f65598995f75 None admin] PATH_INFO: `/v3/users/e0ff5ce14df94fc4a1f5f037539e037c` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.247038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727363bc-e8c4-43d6-8eb6-f65598995f75 None admin] RBAC: Authorizing `identity:get_user(user_id=e0ff5ce14df94fc4a1f5f037539e037c)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.248598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727363bc-e8c4-43d6-8eb6-f65598995f75 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.256253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 38/80] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/users/e0ff5ce14df94fc4a1f5f037539e037c => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:15.256253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727363bc-e8c4-43d6-8eb6-f65598995f75 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.348326 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc0479-fdec-4d8e-9aef-1b41a799d498 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.400077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde24c20-0b72-4993-bf7d-66055cc8a3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.400734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde24c20-0b72-4993-bf7d-66055cc8a3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.401009 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde24c20-0b72-4993-bf7d-66055cc8a3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.401286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde24c20-0b72-4993-bf7d-66055cc8a3f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.411287 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727363bc-e8c4-43d6-8eb6-f65598995f75 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.449701 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef24b08-5bb0-43f2-966a-560cb4a1e1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.450236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef24b08-5bb0-43f2-966a-560cb4a1e1de None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.450520 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef24b08-5bb0-43f2-966a-560cb4a1e1de None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.450797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef24b08-5bb0-43f2-966a-560cb4a1e1de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.471985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde24c20-0b72-4993-bf7d-66055cc8a3f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.473546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde24c20-0b72-4993-bf7d-66055cc8a3f4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.491489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.504527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.536003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef24b08-5bb0-43f2-966a-560cb4a1e1de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.537568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef24b08-5bb0-43f2-966a-560cb4a1e1de None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.591388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 43/81] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:15.592340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde24c20-0b72-4993-bf7d-66055cc8a3f4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.669436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 39/82] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:15.671764 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef24b08-5bb0-43f2-966a-560cb4a1e1de None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.721341 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde24c20-0b72-4993-bf7d-66055cc8a3f4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.737656 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.748760 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.756655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002a27c-052b-4db3-9ce2-9fff89cd98b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.757300 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002a27c-052b-4db3-9ce2-9fff89cd98b4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.759958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002a27c-052b-4db3-9ce2-9fff89cd98b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.759958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002a27c-052b-4db3-9ce2-9fff89cd98b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.770172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002a27c-052b-4db3-9ce2-9fff89cd98b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-1002a27c-052b-4db3-9ce2-9fff89cd98b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:15.775127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:15.776208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:15.777044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:15.777044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:15.777044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:15.777044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:15.777044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:15.777044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:15.777044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:15.777044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:15.777044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:15.777044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:15.777044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:15.777044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:15.777044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 44/83] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:15.778777 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002a27c-052b-4db3-9ce2-9fff89cd98b4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.792362 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef24b08-5bb0-43f2-966a-560cb4a1e1de None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.830268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0023dc-db90-48ee-9105-98474a3dbac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.830889 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0023dc-db90-48ee-9105-98474a3dbac4 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.831174 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0023dc-db90-48ee-9105-98474a3dbac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.831452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0023dc-db90-48ee-9105-98474a3dbac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.892782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002a27c-052b-4db3-9ce2-9fff89cd98b4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.908588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0023dc-db90-48ee-9105-98474a3dbac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.910114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0023dc-db90-48ee-9105-98474a3dbac4 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.948284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4608fc-85aa-4661-a61d-c4ee1b286af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.949418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4608fc-85aa-4661-a61d-c4ee1b286af4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.949929 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4608fc-85aa-4661-a61d-c4ee1b286af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.950428 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4608fc-85aa-4661-a61d-c4ee1b286af4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.952431 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4608fc-85aa-4661-a61d-c4ee1b286af4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.955129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4608fc-85aa-4661-a61d-c4ee1b286af4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.958412 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 45/84] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:15.959632 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4608fc-85aa-4661-a61d-c4ee1b286af4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.995519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:16.030899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:16.042221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 40/85] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:16.043101 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0023dc-db90-48ee-9105-98474a3dbac4 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.115656 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4608fc-85aa-4661-a61d-c4ee1b286af4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.150780 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ad7ae-04c3-4b5c-a14b-5438e2839c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.151417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ad7ae-04c3-4b5c-a14b-5438e2839c8e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.151693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ad7ae-04c3-4b5c-a14b-5438e2839c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.151972 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ad7ae-04c3-4b5c-a14b-5438e2839c8e None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.153156 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ad7ae-04c3-4b5c-a14b-5438e2839c8e None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.154612 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ad7ae-04c3-4b5c-a14b-5438e2839c8e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.156264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 46/86] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:16.157585 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ad7ae-04c3-4b5c-a14b-5438e2839c8e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.169203 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0023dc-db90-48ee-9105-98474a3dbac4 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.210657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088e7ab-310b-4969-9f90-6da64975fceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.211357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088e7ab-310b-4969-9f90-6da64975fceb None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.211649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088e7ab-310b-4969-9f90-6da64975fceb None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.211924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088e7ab-310b-4969-9f90-6da64975fceb None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.213184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088e7ab-310b-4969-9f90-6da64975fceb None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.214855 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088e7ab-310b-4969-9f90-6da64975fceb None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.216580 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 41/87] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:16.217340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088e7ab-310b-4969-9f90-6da64975fceb None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.287964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:16.298001 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ad7ae-04c3-4b5c-a14b-5438e2839c8e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.312544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:16.335771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088e7ab-310b-4969-9f90-6da64975fceb None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.340507 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5222ed-3078-4e39-a089-e3040255f91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.341077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5222ed-3078-4e39-a089-e3040255f91b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.341362 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5222ed-3078-4e39-a089-e3040255f91b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.341640 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5222ed-3078-4e39-a089-e3040255f91b None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.342325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5222ed-3078-4e39-a089-e3040255f91b None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:16.342627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5222ed-3078-4e39-a089-e3040255f91b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.344038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5222ed-3078-4e39-a089-e3040255f91b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.353962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 47/88] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:16.355968 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5222ed-3078-4e39-a089-e3040255f91b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.378823 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138d2b5-7844-4c6a-a5e7-5fa434e682b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.379448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138d2b5-7844-4c6a-a5e7-5fa434e682b4 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.379736 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138d2b5-7844-4c6a-a5e7-5fa434e682b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.380017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138d2b5-7844-4c6a-a5e7-5fa434e682b4 None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.380718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138d2b5-7844-4c6a-a5e7-5fa434e682b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.383269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138d2b5-7844-4c6a-a5e7-5fa434e682b4 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.462007 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5222ed-3078-4e39-a089-e3040255f91b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.471383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 42/89] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:27:16 2022] POST /identity/v3/users => generated 289 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:16.472228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138d2b5-7844-4c6a-a5e7-5fa434e682b4 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.506248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6883d-8b9a-4fc4-93b3-c2e5350c5774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.506915 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6883d-8b9a-4fc4-93b3-c2e5350c5774 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.507200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6883d-8b9a-4fc4-93b3-c2e5350c5774 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.507490 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6883d-8b9a-4fc4-93b3-c2e5350c5774 None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.508873 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6883d-8b9a-4fc4-93b3-c2e5350c5774 None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.510525 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6883d-8b9a-4fc4-93b3-c2e5350c5774 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.518596 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 48/90] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:16.518596 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6883d-8b9a-4fc4-93b3-c2e5350c5774 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.599859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:16.644330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:16.653804 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138d2b5-7844-4c6a-a5e7-5fa434e682b4 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.689354 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6883d-8b9a-4fc4-93b3-c2e5350c5774 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.695353 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b8087-d47c-4f10-bd01-d809ffab85bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.695980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b8087-d47c-4f10-bd01-d809ffab85bb None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.696443 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b8087-d47c-4f10-bd01-d809ffab85bb None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.696731 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b8087-d47c-4f10-bd01-d809ffab85bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.697482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b8087-d47c-4f10-bd01-d809ffab85bb None admin] RBAC: Adding query filter params (role.id=c7b826398b4e466491d4723321aaa5f1, scope.project.id=41be92ad8eba49128ba42ffcedab9caf, user.id=e0ff5ce14df94fc4a1f5f037539e037c) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:16.697795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b8087-d47c-4f10-bd01-d809ffab85bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.699299 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b8087-d47c-4f10-bd01-d809ffab85bb None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.706071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 43/91] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/role_assignments?user.id=e0ff5ce14df94fc4a1f5f037539e037c&scope.project.id=41be92ad8eba49128ba42ffcedab9caf&role.id=c7b826398b4e466491d4723321aaa5f1 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:16.706991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b8087-d47c-4f10-bd01-d809ffab85bb None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.786612 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c53566-f6d6-4dfe-bc20-c9f5bcff3efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.787808 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c53566-f6d6-4dfe-bc20-c9f5bcff3efd None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.788272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c53566-f6d6-4dfe-bc20-c9f5bcff3efd None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.788719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c53566-f6d6-4dfe-bc20-c9f5bcff3efd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.789550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c53566-f6d6-4dfe-bc20-c9f5bcff3efd None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=41be92ad8eba49128ba42ffcedab9caf, user.id=cffc436199314734957ccd7232ed1e29) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:16.789867 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c53566-f6d6-4dfe-bc20-c9f5bcff3efd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.791446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c53566-f6d6-4dfe-bc20-c9f5bcff3efd None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.801787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 49/92] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/role_assignments?user.id=cffc436199314734957ccd7232ed1e29&scope.project.id=41be92ad8eba49128ba42ffcedab9caf&role.id=e7f48343fc1542979ab4459865109e30 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:16.802588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c53566-f6d6-4dfe-bc20-c9f5bcff3efd None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.857158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b8087-d47c-4f10-bd01-d809ffab85bb None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.926965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8110b5f-afcb-4f15-8779-3bedfc585fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.928037 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8110b5f-afcb-4f15-8779-3bedfc585fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.928589 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8110b5f-afcb-4f15-8779-3bedfc585fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.929077 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8110b5f-afcb-4f15-8779-3bedfc585fa4 None admin] PATH_INFO: `/v3/users/e0ff5ce14df94fc4a1f5f037539e037c` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.930857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8110b5f-afcb-4f15-8779-3bedfc585fa4 None admin] RBAC: Authorizing `identity:get_user(user_id=e0ff5ce14df94fc4a1f5f037539e037c)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.932343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8110b5f-afcb-4f15-8779-3bedfc585fa4 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.934462 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 44/93] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/users/e0ff5ce14df94fc4a1f5f037539e037c => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:16.940930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8110b5f-afcb-4f15-8779-3bedfc585fa4 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.948655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c53566-f6d6-4dfe-bc20-c9f5bcff3efd None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.960550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:16.994937 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:17.004113 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a82363-f356-41a2-be5b-dfbc6685e6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:17.005182 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a82363-f356-41a2-be5b-dfbc6685e6ee None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:17.005741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a82363-f356-41a2-be5b-dfbc6685e6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:17.006212 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a82363-f356-41a2-be5b-dfbc6685e6ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:17.007209 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a82363-f356-41a2-be5b-dfbc6685e6ee None admin] RBAC: Adding query filter params (role.id=003cfd128ecf4c238197991ad36f0c9a, scope.project.id=41be92ad8eba49128ba42ffcedab9caf, user.id=e0ff5ce14df94fc4a1f5f037539e037c) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:17.007812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a82363-f356-41a2-be5b-dfbc6685e6ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:17.009838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a82363-f356-41a2-be5b-dfbc6685e6ee None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:17.018412 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 50/94] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/role_assignments?user.id=e0ff5ce14df94fc4a1f5f037539e037c&scope.project.id=41be92ad8eba49128ba42ffcedab9caf&role.id=003cfd128ecf4c238197991ad36f0c9a => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:17.096032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8110b5f-afcb-4f15-8779-3bedfc585fa4 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:17.140670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6961b-1cc7-497e-a228-3e0f2c5588e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:17.141336 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6961b-1cc7-497e-a228-3e0f2c5588e5 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:17.141886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6961b-1cc7-497e-a228-3e0f2c5588e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:17.142170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6961b-1cc7-497e-a228-3e0f2c5588e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:17.224957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6961b-1cc7-497e-a228-3e0f2c5588e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:17.226503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6961b-1cc7-497e-a228-3e0f2c5588e5 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:17.260735 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:17.346307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 45/95] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:27:17.352053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a82363-f356-41a2-be5b-dfbc6685e6ee None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:17.487447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a82363-f356-41a2-be5b-dfbc6685e6ee None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:17.537778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5d7e2-0c65-4124-beb8-d470d3a5aae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:17.538444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5d7e2-0c65-4124-beb8-d470d3a5aae1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:17.538734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5d7e2-0c65-4124-beb8-d470d3a5aae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:17.539015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5d7e2-0c65-4124-beb8-d470d3a5aae1 None admin] PATH_INFO: `/v3/projects/db01eabb387846ca826fb4ef90031208` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:17.541272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d5d7e2-0c65-4124-beb8-d470d3a5aae1 None admin] RBAC: Authorizing `identity:get_project(project_id=db01eabb387846ca826fb4ef90031208)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:17.543408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d5d7e2-0c65-4124-beb8-d470d3a5aae1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:17.547314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 51/96] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:17 2022] GET /identity/v3/projects/db01eabb387846ca826fb4ef90031208 => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:17.553050 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6961b-1cc7-497e-a228-3e0f2c5588e5 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:17.692878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6961b-1cc7-497e-a228-3e0f2c5588e5 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:17.723811 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:17.731792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a64a3-1c99-4e7d-939e-8f42bc444b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:17.732432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a64a3-1c99-4e7d-939e-8f42bc444b1c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:17.732735 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a64a3-1c99-4e7d-939e-8f42bc444b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:17.733075 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a64a3-1c99-4e7d-939e-8f42bc444b1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:17.733888 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a64a3-1c99-4e7d-939e-8f42bc444b1c None admin] RBAC: Adding query filter params (role.id=c7b826398b4e466491d4723321aaa5f1, scope.project.id=db01eabb387846ca826fb4ef90031208, user.id=e0ff5ce14df94fc4a1f5f037539e037c) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:17.734245 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a64a3-1c99-4e7d-939e-8f42bc444b1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:17.736321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a64a3-1c99-4e7d-939e-8f42bc444b1c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:17.742850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 46/97] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:17 2022] GET /identity/v3/role_assignments?user.id=e0ff5ce14df94fc4a1f5f037539e037c&scope.project.id=db01eabb387846ca826fb4ef90031208&role.id=c7b826398b4e466491d4723321aaa5f1 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:19.721689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bdb19-6d79-411d-9048-790eef43b41b None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.722698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bdb19-6d79-411d-9048-790eef43b41b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.723196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bdb19-6d79-411d-9048-790eef43b41b None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.725266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 52/98] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:19.731481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8207af1-9640-4a66-8506-5ddc2ed9aeb5 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.732209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8207af1-9640-4a66-8506-5ddc2ed9aeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.732209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8207af1-9640-4a66-8506-5ddc2ed9aeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.757545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eecaee-ef76-406e-9bca-948ea9f8ba43 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.758209 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eecaee-ef76-406e-9bca-948ea9f8ba43 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.758698 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-b8207af1-9640-4a66-8506-5ddc2ed9aeb5 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:19.759300 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eecaee-ef76-406e-9bca-948ea9f8ba43 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.759947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b8207af1-9640-4a66-8506-5ddc2ed9aeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:19.761611 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 53/99] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:19.770072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c75c7-915c-4034-aa81-c1e2b760ef05 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.770865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c75c7-915c-4034-aa81-c1e2b760ef05 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.771455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c75c7-915c-4034-aa81-c1e2b760ef05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.796046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fd7c75c7-915c-4034-aa81-c1e2b760ef05 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:19.798176 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fd7c75c7-915c-4034-aa81-c1e2b760ef05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:19.842243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:19.848806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 47/100] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:19.886711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 54/101] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:20.128903 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8207af1-9640-4a66-8506-5ddc2ed9aeb5 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:20.246354 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8207af1-9640-4a66-8506-5ddc2ed9aeb5 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:20.283768 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:20.297303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b395ddf-69c6-4f97-b393-4508446c501f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:20.298772 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b395ddf-69c6-4f97-b393-4508446c501f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.299058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b395ddf-69c6-4f97-b393-4508446c501f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.299338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b395ddf-69c6-4f97-b393-4508446c501f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.316825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b395ddf-69c6-4f97-b393-4508446c501f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-5b395ddf-69c6-4f97-b393-4508446c501f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:20.322199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:20.323139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:20.323917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:20.323917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:20.323917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:20.323917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:20.323917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:20.323917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:20.323917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:20.323917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:20.323917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:20.323917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:20.323917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:20.323917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:20.323917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 48/102] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:20.329787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c75c7-915c-4034-aa81-c1e2b760ef05 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:20.361509 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b395ddf-69c6-4f97-b393-4508446c501f None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:20.554818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c75c7-915c-4034-aa81-c1e2b760ef05 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:20.591435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b395ddf-69c6-4f97-b393-4508446c501f None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:20.612990 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d287b8-fe24-4ab5-8ee6-a9dddfc836d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:20.613635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d287b8-fe24-4ab5-8ee6-a9dddfc836d2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.613976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d287b8-fe24-4ab5-8ee6-a9dddfc836d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.614256 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d287b8-fe24-4ab5-8ee6-a9dddfc836d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.616112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d287b8-fe24-4ab5-8ee6-a9dddfc836d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:20.616460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d287b8-fe24-4ab5-8ee6-a9dddfc836d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:20.620188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d287b8-fe24-4ab5-8ee6-a9dddfc836d2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:20.639416 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 55/103] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:20.640759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f6ce7-1616-48de-8ba6-a27c41e4a9d5 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.641087 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f6ce7-1616-48de-8ba6-a27c41e4a9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.641393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f6ce7-1616-48de-8ba6-a27c41e4a9d5 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.651535 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 56/104] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:20 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:20.652472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f6ce7-1616-48de-8ba6-a27c41e4a9d5 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:20.677865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ea072-1428-4a69-9db1-8b9f1bf5d10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:20.678573 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ea072-1428-4a69-9db1-8b9f1bf5d10e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.679097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ea072-1428-4a69-9db1-8b9f1bf5d10e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.679451 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ea072-1428-4a69-9db1-8b9f1bf5d10e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.694482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ea072-1428-4a69-9db1-8b9f1bf5d10e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-ea8ea072-1428-4a69-9db1-8b9f1bf5d10e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:20.701267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:20.702372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:20.703122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:20.703122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:20.703122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:20.703122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:20.703122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:20.703122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:20.703122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:20.703122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:20.703122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:20.703122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:20.703122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:20.703122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:20.711376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 49/105] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:20.711376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009afc8f-b9f7-4238-a635-dd34e4d6aa3c None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.711376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009afc8f-b9f7-4238-a635-dd34e4d6aa3c None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.711376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009afc8f-b9f7-4238-a635-dd34e4d6aa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.733130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-009afc8f-b9f7-4238-a635-dd34e4d6aa3c None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:20.735629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-009afc8f-b9f7-4238-a635-dd34e4d6aa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:20.754646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:20.781452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:20.815445 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 50/106] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:20.824610 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009afc8f-b9f7-4238-a635-dd34e4d6aa3c None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:20.864797 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f6ce7-1616-48de-8ba6-a27c41e4a9d5 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:20.926720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149fa90-3774-418b-874a-ed71dee4c2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:20.928888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149fa90-3774-418b-874a-ed71dee4c2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.929980 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149fa90-3774-418b-874a-ed71dee4c2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.930697 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149fa90-3774-418b-874a-ed71dee4c2d6 None admin] PATH_INFO: `/v3/users/cffc436199314734957ccd7232ed1e29` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.934123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149fa90-3774-418b-874a-ed71dee4c2d6 None admin] RBAC: Authorizing `identity:get_user(user_id=cffc436199314734957ccd7232ed1e29)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:20.938200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149fa90-3774-418b-874a-ed71dee4c2d6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:20.943772 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 57/107] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:20 2022] GET /identity/v3/users/cffc436199314734957ccd7232ed1e29 => generated 247 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:20.946307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cf99a-ec76-4774-8570-42ec5aeea0ef None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.947009 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cf99a-ec76-4774-8570-42ec5aeea0ef None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.947673 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cf99a-ec76-4774-8570-42ec5aeea0ef None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.950529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 58/108] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:20.952981 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345af0c-45e3-44a6-be9e-47fc93fc03ef None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.953275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345af0c-45e3-44a6-be9e-47fc93fc03ef None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.954072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345af0c-45e3-44a6-be9e-47fc93fc03ef None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.955620 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 59/109] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:20.961629 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345af0c-45e3-44a6-be9e-47fc93fc03ef None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:21.019485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009afc8f-b9f7-4238-a635-dd34e4d6aa3c None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:21.060561 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d910543-415c-4fec-a715-ed8585f77767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:21.061227 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d910543-415c-4fec-a715-ed8585f77767 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.061507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d910543-415c-4fec-a715-ed8585f77767 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.061786 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d910543-415c-4fec-a715-ed8585f77767 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.062480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d910543-415c-4fec-a715-ed8585f77767 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:21.062786 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d910543-415c-4fec-a715-ed8585f77767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:21.064259 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d910543-415c-4fec-a715-ed8585f77767 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:21.074187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 51/110] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:21.075767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4474a-c627-4450-a8c6-130cf8116da5 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.076208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4474a-c627-4450-a8c6-130cf8116da5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.076715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4474a-c627-4450-a8c6-130cf8116da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.103035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-c1e4474a-c627-4450-a8c6-130cf8116da5 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:21.105164 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c1e4474a-c627-4450-a8c6-130cf8116da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:21.110187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:21.115919 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345af0c-45e3-44a6-be9e-47fc93fc03ef None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:21.141043 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:21.178495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82642f73-1e2f-4f2a-8a76-f9a030c6c0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:21.179494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82642f73-1e2f-4f2a-8a76-f9a030c6c0cc None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.179945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82642f73-1e2f-4f2a-8a76-f9a030c6c0cc None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.180384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82642f73-1e2f-4f2a-8a76-f9a030c6c0cc None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.182384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82642f73-1e2f-4f2a-8a76-f9a030c6c0cc None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:21.185036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82642f73-1e2f-4f2a-8a76-f9a030c6c0cc None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:21.186573 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 52/111] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:21.196575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210cc77-d78f-4b7b-81a2-e6ca2c2b7f39 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.196878 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 60/112] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:20 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:21.197025 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210cc77-d78f-4b7b-81a2-e6ca2c2b7f39 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.197025 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210cc77-d78f-4b7b-81a2-e6ca2c2b7f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.197267 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfc912-f806-4186-81ba-9749f304985d None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.197267 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfc912-f806-4186-81ba-9749f304985d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.197267 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfc912-f806-4186-81ba-9749f304985d None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.197267 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 61/113] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:21.197267 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfc912-f806-4186-81ba-9749f304985d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:21.218847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-8210cc77-d78f-4b7b-81a2-e6ca2c2b7f39 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:21.220269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-8210cc77-d78f-4b7b-81a2-e6ca2c2b7f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:21.304355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 53/114] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:21.305259 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210cc77-d78f-4b7b-81a2-e6ca2c2b7f39 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:21.372978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfc912-f806-4186-81ba-9749f304985d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:21.456295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db7af2-dc7b-491d-b718-2876bc529057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:21.457615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:21.458692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db7af2-dc7b-491d-b718-2876bc529057 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.460351 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db7af2-dc7b-491d-b718-2876bc529057 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.461131 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db7af2-dc7b-491d-b718-2876bc529057 None admin] PATH_INFO: `/v3/users/bb18b28bdf89484190c796b9fa8b60be` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.471546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210cc77-d78f-4b7b-81a2-e6ca2c2b7f39 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:21.484166 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db7af2-dc7b-491d-b718-2876bc529057 None admin] RBAC: Authorizing `identity:get_user(user_id=bb18b28bdf89484190c796b9fa8b60be)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:21.487172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db7af2-dc7b-491d-b718-2876bc529057 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:21.490034 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 62/115] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:21 2022] GET /identity/v3/users/bb18b28bdf89484190c796b9fa8b60be => generated 289 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:21.493058 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db7af2-dc7b-491d-b718-2876bc529057 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:21.534271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:21.534897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e9565-2a0c-4213-9103-9b94736437fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:21.536197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e9565-2a0c-4213-9103-9b94736437fe None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.536701 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e9565-2a0c-4213-9103-9b94736437fe None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.537105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e9565-2a0c-4213-9103-9b94736437fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.540477 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e9565-2a0c-4213-9103-9b94736437fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:21.541649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e9565-2a0c-4213-9103-9b94736437fe None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:21.542737 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 54/116] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:21.546041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fcb80-553b-497e-a9b7-62c7be7638da None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.546041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fcb80-553b-497e-a9b7-62c7be7638da None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.546041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fcb80-553b-497e-a9b7-62c7be7638da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.567739 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-102fcb80-553b-497e-a9b7-62c7be7638da None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:21.568994 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-102fcb80-553b-497e-a9b7-62c7be7638da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:21.648828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 55/117] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:21.649611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fcb80-553b-497e-a9b7-62c7be7638da None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:21.677179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db7af2-dc7b-491d-b718-2876bc529057 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:21.731649 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee32c3c-5ca6-4d32-9113-c638e09a3030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:21.732327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee32c3c-5ca6-4d32-9113-c638e09a3030 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.732662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee32c3c-5ca6-4d32-9113-c638e09a3030 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.732919 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee32c3c-5ca6-4d32-9113-c638e09a3030 None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf/users/cffc436199314734957ccd7232ed1e29/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.735433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee32c3c-5ca6-4d32-9113-c638e09a3030 None admin] RBAC: Authorizing `identity:create_grant(project_id=41be92ad8eba49128ba42ffcedab9caf, user_id=cffc436199314734957ccd7232ed1e29, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:21.736947 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee32c3c-5ca6-4d32-9113-c638e09a3030 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:21.767804 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:21.777066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fcb80-553b-497e-a9b7-62c7be7638da None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:21.811471 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1128e-cf60-41a4-85f7-30dfcf90cbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:21.812192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1128e-cf60-41a4-85f7-30dfcf90cbb7 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.812429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1128e-cf60-41a4-85f7-30dfcf90cbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.812683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1128e-cf60-41a4-85f7-30dfcf90cbb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.827671 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1128e-cf60-41a4-85f7-30dfcf90cbb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-50b1128e-cf60-41a4-85f7-30dfcf90cbb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:21.832792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:21.834252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:21.835022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:21.835022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:21.835022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:21.835022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:21.835022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:21.835022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:21.835022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:21.835022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:21.835022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:21.835022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:21.835022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:21.835022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:21.835022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 56/118] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:21.835022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1128e-cf60-41a4-85f7-30dfcf90cbb7 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:21.874867 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 63/119] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:21 2022] PUT /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf/users/cffc436199314734957ccd7232ed1e29/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:27:21.875660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee32c3c-5ca6-4d32-9113-c638e09a3030 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:22.012028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1128e-cf60-41a4-85f7-30dfcf90cbb7 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:22.034068 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:22.054809 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee32c3c-5ca6-4d32-9113-c638e09a3030 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:22.063731 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47724bdd-536c-4cf2-8522-aad8beec2311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:22.064690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47724bdd-536c-4cf2-8522-aad8beec2311 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.065124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47724bdd-536c-4cf2-8522-aad8beec2311 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.065996 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47724bdd-536c-4cf2-8522-aad8beec2311 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.093375 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbb143-d592-47b0-95e6-453ba1abffb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:22.095176 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbb143-d592-47b0-95e6-453ba1abffb9 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.095176 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbb143-d592-47b0-95e6-453ba1abffb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.095176 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbb143-d592-47b0-95e6-453ba1abffb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.096052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fbb143-d592-47b0-95e6-453ba1abffb9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:22.097239 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fbb143-d592-47b0-95e6-453ba1abffb9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:22.102488 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:22.158878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47724bdd-536c-4cf2-8522-aad8beec2311 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:22.160389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47724bdd-536c-4cf2-8522-aad8beec2311 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:22.193661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 64/120] 10.222.0.70 () {60 vars in 1141 bytes} [Wed Aug 24 16:27:21 2022] POST /identity/v3/projects => generated 302 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:22.195036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbb143-d592-47b0-95e6-453ba1abffb9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:22.360931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 57/121] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:22.362781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47724bdd-536c-4cf2-8522-aad8beec2311 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:22.375098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbb143-d592-47b0-95e6-453ba1abffb9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:22.419443 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead8b95-b9da-447a-81e1-60752b4ad27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:22.420249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead8b95-b9da-447a-81e1-60752b4ad27a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.420716 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead8b95-b9da-447a-81e1-60752b4ad27a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.421213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead8b95-b9da-447a-81e1-60752b4ad27a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.424490 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:22.436108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead8b95-b9da-447a-81e1-60752b4ad27a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-aead8b95-b9da-447a-81e1-60752b4ad27a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:22.442721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:22.443775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:22.444554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:22.444554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:22.444554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:22.444554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:22.444554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:22.444554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:22.444554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:22.444554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:22.444554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:22.444554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:22.444554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:22.444554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:22.446420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 65/122] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:22.447019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead8b95-b9da-447a-81e1-60752b4ad27a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:22.472272 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:22.534887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47724bdd-536c-4cf2-8522-aad8beec2311 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:22.586455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809bd29-b036-49eb-a40f-77138eaf0bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:22.587663 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809bd29-b036-49eb-a40f-77138eaf0bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.588591 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809bd29-b036-49eb-a40f-77138eaf0bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.589021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809bd29-b036-49eb-a40f-77138eaf0bb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.590160 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809bd29-b036-49eb-a40f-77138eaf0bb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:22.590679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809bd29-b036-49eb-a40f-77138eaf0bb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:22.593351 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809bd29-b036-49eb-a40f-77138eaf0bb6 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:22.599187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead8b95-b9da-447a-81e1-60752b4ad27a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:22.606511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 58/123] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:22.607780 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809bd29-b036-49eb-a40f-77138eaf0bb6 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:22.643049 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964f847-7071-4e12-8e88-9f595d78a173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:22.644016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964f847-7071-4e12-8e88-9f595d78a173 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.644544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964f847-7071-4e12-8e88-9f595d78a173 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.644967 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964f847-7071-4e12-8e88-9f595d78a173 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.658036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964f847-7071-4e12-8e88-9f595d78a173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-4964f847-7071-4e12-8e88-9f595d78a173 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:22.665554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:22.671303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:22.672082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:22.672082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:22.672082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:22.672082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:22.672082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:22.672082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:22.672082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:22.672082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:22.672082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:22.672082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:22.672082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:27:22.672082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:22.672082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 66/124] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:22.672082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964f847-7071-4e12-8e88-9f595d78a173 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:22.737541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:22.757557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809bd29-b036-49eb-a40f-77138eaf0bb6 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:22.792052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964f847-7071-4e12-8e88-9f595d78a173 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:22.796617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:22.835154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec574fd-fe9c-4e0e-b457-512814b877ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:22.837121 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec574fd-fe9c-4e0e-b457-512814b877ed None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.838227 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec574fd-fe9c-4e0e-b457-512814b877ed None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.838961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec574fd-fe9c-4e0e-b457-512814b877ed None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.841346 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec574fd-fe9c-4e0e-b457-512814b877ed None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:22.842816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7d23e-42df-436d-b2f2-a2f6bc836486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:22.843547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7d23e-42df-436d-b2f2-a2f6bc836486 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.843907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7d23e-42df-436d-b2f2-a2f6bc836486 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.844298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7d23e-42df-436d-b2f2-a2f6bc836486 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.844696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec574fd-fe9c-4e0e-b457-512814b877ed None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:22.845163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7d23e-42df-436d-b2f2-a2f6bc836486 None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:22.845547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7d23e-42df-436d-b2f2-a2f6bc836486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:22.847050 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7d23e-42df-436d-b2f2-a2f6bc836486 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:22.848239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 59/125] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:22 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:22.854319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec574fd-fe9c-4e0e-b457-512814b877ed None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:22.856786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 67/126] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:22.857664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7d23e-42df-436d-b2f2-a2f6bc836486 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:23.007740 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7d23e-42df-436d-b2f2-a2f6bc836486 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:23.014026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec574fd-fe9c-4e0e-b457-512814b877ed None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:23.029096 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:23.048392 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4fe5f-8a74-43fa-a190-7ee6a4876b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:23.049104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4fe5f-8a74-43fa-a190-7ee6a4876b55 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:23.049393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4fe5f-8a74-43fa-a190-7ee6a4876b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:23.049667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4fe5f-8a74-43fa-a190-7ee6a4876b55 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:23.050442 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4fe5f-8a74-43fa-a190-7ee6a4876b55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:23.050659 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4fe5f-8a74-43fa-a190-7ee6a4876b55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:23.052130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4fe5f-8a74-43fa-a190-7ee6a4876b55 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:23.060205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe041de-70d5-4e54-a8ba-ca5e742e432c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:23.061163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe041de-70d5-4e54-a8ba-ca5e742e432c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:23.061596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe041de-70d5-4e54-a8ba-ca5e742e432c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:23.062042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe041de-70d5-4e54-a8ba-ca5e742e432c None admin] PATH_INFO: `/v3/users/e0ff5ce14df94fc4a1f5f037539e037c` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:23.065118 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe041de-70d5-4e54-a8ba-ca5e742e432c None admin] RBAC: Authorizing `identity:get_user(user_id=e0ff5ce14df94fc4a1f5f037539e037c)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:23.066973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 68/127] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:23.067494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe041de-70d5-4e54-a8ba-ca5e742e432c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:23.067958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4fe5f-8a74-43fa-a190-7ee6a4876b55 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:23.069655 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 60/128] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:22 2022] GET /identity/v3/users/e0ff5ce14df94fc4a1f5f037539e037c => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:23.070596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe041de-70d5-4e54-a8ba-ca5e742e432c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:23.130513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:23.183675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4fe5f-8a74-43fa-a190-7ee6a4876b55 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:23.192095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe041de-70d5-4e54-a8ba-ca5e742e432c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:23.221344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf124f8-6a2c-4bf6-ae4c-66283230e549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:23.221983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf124f8-6a2c-4bf6-ae4c-66283230e549 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:23.222270 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf124f8-6a2c-4bf6-ae4c-66283230e549 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:23.222615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf124f8-6a2c-4bf6-ae4c-66283230e549 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:23.223387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf124f8-6a2c-4bf6-ae4c-66283230e549 None admin] RBAC: Adding query filter params (role.id=a915b705941d46538bd6a59aac287977, scope.project.id=41be92ad8eba49128ba42ffcedab9caf, user.id=bb18b28bdf89484190c796b9fa8b60be) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:23.223706 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf124f8-6a2c-4bf6-ae4c-66283230e549 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:23.225310 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf124f8-6a2c-4bf6-ae4c-66283230e549 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:23.231438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 69/129] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:23 2022] GET /identity/v3/role_assignments?user.id=bb18b28bdf89484190c796b9fa8b60be&scope.project.id=41be92ad8eba49128ba42ffcedab9caf&role.id=a915b705941d46538bd6a59aac287977 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:23.232317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf124f8-6a2c-4bf6-ae4c-66283230e549 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:23.233843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a3bad-3d92-441d-8df2-01b6941d839c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:23.235263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a3bad-3d92-441d-8df2-01b6941d839c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:23.235709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a3bad-3d92-441d-8df2-01b6941d839c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:23.236130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a3bad-3d92-441d-8df2-01b6941d839c None admin] PATH_INFO: `/v3/users/e0ff5ce14df94fc4a1f5f037539e037c` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:23.237678 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a3bad-3d92-441d-8df2-01b6941d839c None admin] RBAC: Authorizing `identity:get_user(user_id=e0ff5ce14df94fc4a1f5f037539e037c)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:23.239402 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a3bad-3d92-441d-8df2-01b6941d839c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:23.241408 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 61/130] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:23 2022] GET /identity/v3/users/e0ff5ce14df94fc4a1f5f037539e037c => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:23.247288 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a3bad-3d92-441d-8df2-01b6941d839c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:23.327398 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:23.410137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf124f8-6a2c-4bf6-ae4c-66283230e549 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:23.434275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a3bad-3d92-441d-8df2-01b6941d839c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:23.449923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbb10b-2c92-4fbb-b768-1eff035dd52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:23.450603 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbb10b-2c92-4fbb-b768-1eff035dd52d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:23.450893 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbb10b-2c92-4fbb-b768-1eff035dd52d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:23.451174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbb10b-2c92-4fbb-b768-1eff035dd52d None admin] PATH_INFO: `/v3/users/e0ff5ce14df94fc4a1f5f037539e037c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:23.452620 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbb10b-2c92-4fbb-b768-1eff035dd52d None admin] RBAC: Authorizing `identity:get_user(user_id=e0ff5ce14df94fc4a1f5f037539e037c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:23.454171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbb10b-2c92-4fbb-b768-1eff035dd52d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:23.456362 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 70/131] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:23 2022] GET /identity/v3/users/e0ff5ce14df94fc4a1f5f037539e037c => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:23.461304 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbb10b-2c92-4fbb-b768-1eff035dd52d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:23.474459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:23.508911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc50de3-1515-4beb-8036-32b4e59e006c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:23.510193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc50de3-1515-4beb-8036-32b4e59e006c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:23.510763 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc50de3-1515-4beb-8036-32b4e59e006c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:23.511464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc50de3-1515-4beb-8036-32b4e59e006c None admin] PATH_INFO: `/v3/projects/db01eabb387846ca826fb4ef90031208` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:23.520456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc50de3-1515-4beb-8036-32b4e59e006c None admin] RBAC: Authorizing `identity:get_project(project_id=db01eabb387846ca826fb4ef90031208)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:23.524456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc50de3-1515-4beb-8036-32b4e59e006c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:23.525214 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 62/132] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:23 2022] GET /identity/v3/projects/db01eabb387846ca826fb4ef90031208 => generated 312 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:23.526497 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc50de3-1515-4beb-8036-32b4e59e006c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:23.682137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbb10b-2c92-4fbb-b768-1eff035dd52d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:23.710649 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:23.757359 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e1fc0-e675-4a0b-b797-4fbd31286c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:23.758019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e1fc0-e675-4a0b-b797-4fbd31286c6d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:23.758302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e1fc0-e675-4a0b-b797-4fbd31286c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:23.758577 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e1fc0-e675-4a0b-b797-4fbd31286c6d None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:23.760041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e1fc0-e675-4a0b-b797-4fbd31286c6d None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:23.761582 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e1fc0-e675-4a0b-b797-4fbd31286c6d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:23.763677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 71/133] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:23 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:23.764218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e1fc0-e675-4a0b-b797-4fbd31286c6d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:23.790180 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc50de3-1515-4beb-8036-32b4e59e006c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:23.878765 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6085475-7ed5-4635-bafc-2a09c285bcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:23.879772 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6085475-7ed5-4635-bafc-2a09c285bcfd None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:23.879772 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6085475-7ed5-4635-bafc-2a09c285bcfd None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:23.879988 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6085475-7ed5-4635-bafc-2a09c285bcfd None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:23.884760 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6085475-7ed5-4635-bafc-2a09c285bcfd None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:23.886253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6085475-7ed5-4635-bafc-2a09c285bcfd None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:23.888297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 63/134] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:23 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:23.898413 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6085475-7ed5-4635-bafc-2a09c285bcfd None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:23.925951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e1fc0-e675-4a0b-b797-4fbd31286c6d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:24.004500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8594886-54b9-4871-874b-ca911b2032ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:24.006002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8594886-54b9-4871-874b-ca911b2032ef None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:24.006293 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8594886-54b9-4871-874b-ca911b2032ef None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:24.006619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8594886-54b9-4871-874b-ca911b2032ef None admin] PATH_INFO: `/v3/projects/db01eabb387846ca826fb4ef90031208/users/e0ff5ce14df94fc4a1f5f037539e037c/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:24.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8594886-54b9-4871-874b-ca911b2032ef None admin] RBAC: Authorizing `identity:create_grant(project_id=db01eabb387846ca826fb4ef90031208, user_id=e0ff5ce14df94fc4a1f5f037539e037c, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:24.013465 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8594886-54b9-4871-874b-ca911b2032ef None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:24.020880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:24.082518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6085475-7ed5-4635-bafc-2a09c285bcfd None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:24.121226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96c69a-571e-4d95-b229-ffe0e1acb187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:24.121962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96c69a-571e-4d95-b229-ffe0e1acb187 None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:24.122246 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96c69a-571e-4d95-b229-ffe0e1acb187 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:24.122524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96c69a-571e-4d95-b229-ffe0e1acb187 None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf/users/e0ff5ce14df94fc4a1f5f037539e037c/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:24.124574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96c69a-571e-4d95-b229-ffe0e1acb187 None admin] RBAC: Authorizing `identity:create_grant(project_id=41be92ad8eba49128ba42ffcedab9caf, user_id=e0ff5ce14df94fc4a1f5f037539e037c, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:24.126638 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96c69a-571e-4d95-b229-ffe0e1acb187 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:24.134022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 72/135] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:23 2022] PUT /identity/v3/projects/db01eabb387846ca826fb4ef90031208/users/e0ff5ce14df94fc4a1f5f037539e037c/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:27:24.135000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8594886-54b9-4871-874b-ca911b2032ef None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:24.215099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:24.274790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 64/136] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:23 2022] PUT /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf/users/e0ff5ce14df94fc4a1f5f037539e037c/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:27:24.302824 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8594886-54b9-4871-874b-ca911b2032ef None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:24.356968 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b2624-6de3-4104-9939-16da3a46e7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:24.357997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b2624-6de3-4104-9939-16da3a46e7ae None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:24.358479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b2624-6de3-4104-9939-16da3a46e7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:24.358946 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b2624-6de3-4104-9939-16da3a46e7ae None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf/users/e0ff5ce14df94fc4a1f5f037539e037c/roles/003cfd128ecf4c238197991ad36f0c9a` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:24.366857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b2624-6de3-4104-9939-16da3a46e7ae None admin] RBAC: Authorizing `identity:create_grant(project_id=41be92ad8eba49128ba42ffcedab9caf, user_id=e0ff5ce14df94fc4a1f5f037539e037c, role_id=003cfd128ecf4c238197991ad36f0c9a)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:24.368687 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b2624-6de3-4104-9939-16da3a46e7ae None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:24.452677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 73/137] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:24 2022] PUT /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf/users/e0ff5ce14df94fc4a1f5f037539e037c/roles/003cfd128ecf4c238197991ad36f0c9a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:27:24.576579 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376aa0cc-41a4-4cb2-8b22-238f7eba8fb9 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:24.576873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376aa0cc-41a4-4cb2-8b22-238f7eba8fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:24.577152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376aa0cc-41a4-4cb2-8b22-238f7eba8fb9 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:24.578748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 65/138] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:24.585173 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3e00a-595e-4ac9-a478-8af09d034b7e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:24.585469 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3e00a-595e-4ac9-a478-8af09d034b7e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:24.585756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3e00a-595e-4ac9-a478-8af09d034b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:24.608686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-adf3e00a-595e-4ac9-a478-8af09d034b7e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:24.610587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-adf3e00a-595e-4ac9-a478-8af09d034b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:24.698079 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 74/139] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:24.861471 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3e4f5-d1d7-4016-81ea-a89eaefaa8ac None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:24.861988 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3e4f5-d1d7-4016-81ea-a89eaefaa8ac None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:24.862131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3e4f5-d1d7-4016-81ea-a89eaefaa8ac None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:24.863518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 66/140] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:24.869534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768935e-dea8-4c07-b015-d255a0a0e102 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:24.869943 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768935e-dea8-4c07-b015-d255a0a0e102 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:24.870084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768935e-dea8-4c07-b015-d255a0a0e102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:24.897786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7768935e-dea8-4c07-b015-d255a0a0e102 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:24.899223 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7768935e-dea8-4c07-b015-d255a0a0e102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:24.930084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:24.990925 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 75/141] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:25.022410 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3e4f5-d1d7-4016-81ea-a89eaefaa8ac None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.146874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3e4f5-d1d7-4016-81ea-a89eaefaa8ac None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:25.164094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:25.188986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c815799-8b57-4398-9f87-86c0ce662027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:25.190080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c815799-8b57-4398-9f87-86c0ce662027 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.190616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c815799-8b57-4398-9f87-86c0ce662027 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.191135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c815799-8b57-4398-9f87-86c0ce662027 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.206195 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c815799-8b57-4398-9f87-86c0ce662027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-6c815799-8b57-4398-9f87-86c0ce662027 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:25.211668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:25.212699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:25.213460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:25.213460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:25.213460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:25.213460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:25.213460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:25.213460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:25.213460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:25.213460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:25.213460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:25.213460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:25.213460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:25.213460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:25.215928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 67/142] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:25.223345 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768935e-dea8-4c07-b015-d255a0a0e102 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.373895 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768935e-dea8-4c07-b015-d255a0a0e102 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:25.438395 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041efb6d-5ac3-4e9c-a93d-b37c81c1c96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:25.438963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041efb6d-5ac3-4e9c-a93d-b37c81c1c96a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.439244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041efb6d-5ac3-4e9c-a93d-b37c81c1c96a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.439511 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041efb6d-5ac3-4e9c-a93d-b37c81c1c96a None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.440209 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041efb6d-5ac3-4e9c-a93d-b37c81c1c96a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:25.440546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041efb6d-5ac3-4e9c-a93d-b37c81c1c96a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:25.442029 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041efb6d-5ac3-4e9c-a93d-b37c81c1c96a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:25.454269 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 76/143] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:25.460151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c815799-8b57-4398-9f87-86c0ce662027 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.549835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041efb6d-5ac3-4e9c-a93d-b37c81c1c96a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.598651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c815799-8b57-4398-9f87-86c0ce662027 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:25.628832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:25.656459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686d1fb-ea5c-4235-bd77-aed9894a146f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:25.656459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686d1fb-ea5c-4235-bd77-aed9894a146f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.656459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686d1fb-ea5c-4235-bd77-aed9894a146f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.656459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686d1fb-ea5c-4235-bd77-aed9894a146f None admin] PATH_INFO: `/v3/users/cffc436199314734957ccd7232ed1e29` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.657045 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686d1fb-ea5c-4235-bd77-aed9894a146f None admin] RBAC: Authorizing `identity:get_user(user_id=cffc436199314734957ccd7232ed1e29)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:25.659420 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686d1fb-ea5c-4235-bd77-aed9894a146f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:25.660348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 68/144] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:25 2022] GET /identity/v3/users/cffc436199314734957ccd7232ed1e29 => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:25.667158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686d1fb-ea5c-4235-bd77-aed9894a146f None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.727333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:25.774486 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041efb6d-5ac3-4e9c-a93d-b37c81c1c96a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:25.818727 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686d1fb-ea5c-4235-bd77-aed9894a146f None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:25.846290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f42109-6487-488e-bbfe-9139b6175ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:25.847336 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f42109-6487-488e-bbfe-9139b6175ebe None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.847810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f42109-6487-488e-bbfe-9139b6175ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.848281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f42109-6487-488e-bbfe-9139b6175ebe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.853353 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f42109-6487-488e-bbfe-9139b6175ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:25.855249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f42109-6487-488e-bbfe-9139b6175ebe None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:25.857254 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 77/145] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:25.862994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f42109-6487-488e-bbfe-9139b6175ebe None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.896651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa98cfa-0332-423f-81a2-0888e1a5ee09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:25.896873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa98cfa-0332-423f-81a2-0888e1a5ee09 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.900128 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa98cfa-0332-423f-81a2-0888e1a5ee09 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.900128 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa98cfa-0332-423f-81a2-0888e1a5ee09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.064544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:26.064713 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa98cfa-0332-423f-81a2-0888e1a5ee09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.065510 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa98cfa-0332-423f-81a2-0888e1a5ee09 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.073082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f42109-6487-488e-bbfe-9139b6175ebe None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.140111 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02859be6-d980-4aa2-9ce8-3514b54dc332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.141126 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02859be6-d980-4aa2-9ce8-3514b54dc332 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.141892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02859be6-d980-4aa2-9ce8-3514b54dc332 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.142338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02859be6-d980-4aa2-9ce8-3514b54dc332 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.143190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02859be6-d980-4aa2-9ce8-3514b54dc332 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.145008 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02859be6-d980-4aa2-9ce8-3514b54dc332 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.221095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:26.263276 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 78/146] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:27:25 2022] POST /identity/v3/users => generated 283 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:26.322200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 69/147] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:27:26.328021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02859be6-d980-4aa2-9ce8-3514b54dc332 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:26.484185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02859be6-d980-4aa2-9ce8-3514b54dc332 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.489896 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedab863-6a28-40b1-8108-ba259695efca None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.490562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedab863-6a28-40b1-8108-ba259695efca None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.491137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedab863-6a28-40b1-8108-ba259695efca None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.493392 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 70/148] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:26.502043 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc3ed9-52a6-40cf-a74b-ed7f678e08ff None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.502875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc3ed9-52a6-40cf-a74b-ed7f678e08ff None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.503551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc3ed9-52a6-40cf-a74b-ed7f678e08ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.525798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661c4ce-dcac-43a6-94da-c3707e12a952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.527541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661c4ce-dcac-43a6-94da-c3707e12a952 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.527541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661c4ce-dcac-43a6-94da-c3707e12a952 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.527541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661c4ce-dcac-43a6-94da-c3707e12a952 None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.528278 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661c4ce-dcac-43a6-94da-c3707e12a952 None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.530622 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661c4ce-dcac-43a6-94da-c3707e12a952 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.533041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 79/149] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:26 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:26.540441 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661c4ce-dcac-43a6-94da-c3707e12a952 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:26.556582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-1afc3ed9-52a6-40cf-a74b-ed7f678e08ff None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:26.558337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1afc3ed9-52a6-40cf-a74b-ed7f678e08ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:26.564820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:26.670677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661c4ce-dcac-43a6-94da-c3707e12a952 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.716661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a327265-3cb5-46ca-ab0f-ad14f85a76d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.717444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a327265-3cb5-46ca-ab0f-ad14f85a76d4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.717788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a327265-3cb5-46ca-ab0f-ad14f85a76d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.718127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a327265-3cb5-46ca-ab0f-ad14f85a76d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.720475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a327265-3cb5-46ca-ab0f-ad14f85a76d4 None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=41be92ad8eba49128ba42ffcedab9caf, user.id=cffc436199314734957ccd7232ed1e29) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:26.720923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a327265-3cb5-46ca-ab0f-ad14f85a76d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.722573 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a327265-3cb5-46ca-ab0f-ad14f85a76d4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.737223 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 80/150] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:26 2022] GET /identity/v3/role_assignments?user.id=cffc436199314734957ccd7232ed1e29&scope.project.id=41be92ad8eba49128ba42ffcedab9caf&role.id=e7f48343fc1542979ab4459865109e30 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:26.817677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 71/151] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:27.157172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1516a-2528-4bf8-95ed-53755eb8b70f None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:27.157677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1516a-2528-4bf8-95ed-53755eb8b70f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:27.158075 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1516a-2528-4bf8-95ed-53755eb8b70f None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:27.159713 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 81/152] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:27.166829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828e422-c0c5-4328-8c70-396bf6e8783a None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:27.168039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828e422-c0c5-4328-8c70-396bf6e8783a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:27.168569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828e422-c0c5-4328-8c70-396bf6e8783a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:27.205566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-b828e422-c0c5-4328-8c70-396bf6e8783a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:27.207422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b828e422-c0c5-4328-8c70-396bf6e8783a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:27.302591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1516a-2528-4bf8-95ed-53755eb8b70f None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:27.363895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 72/153] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:27.430554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:27.463606 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1516a-2528-4bf8-95ed-53755eb8b70f None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:27.512141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82d029-294f-4087-9253-52edff3cbe1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:27.512830 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82d029-294f-4087-9253-52edff3cbe1e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:27.513115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82d029-294f-4087-9253-52edff3cbe1e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:27.513389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82d029-294f-4087-9253-52edff3cbe1e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:27.530400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82d029-294f-4087-9253-52edff3cbe1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-ca82d029-294f-4087-9253-52edff3cbe1e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:27.535636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:27.537307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:27.539952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:27.539952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:27.539952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:27.539952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:27.539952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:27.539952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:27.539952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:27.539952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:27.539952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:27.539952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:27.539952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:27.539952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:27.539952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 82/154] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:27.551006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828e422-c0c5-4328-8c70-396bf6e8783a None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:27.560480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:27.752803 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828e422-c0c5-4328-8c70-396bf6e8783a None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:27.761522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad695a9-fda5-4226-bb38-4da5b961744c None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:27.762336 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad695a9-fda5-4226-bb38-4da5b961744c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:27.762827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad695a9-fda5-4226-bb38-4da5b961744c None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:27.765731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 83/155] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:27.776822 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f627d3f-d7f1-4035-b139-7cea922e7093 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:27.777615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f627d3f-d7f1-4035-b139-7cea922e7093 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:27.778680 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f627d3f-d7f1-4035-b139-7cea922e7093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:27.806239 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4f627d3f-d7f1-4035-b139-7cea922e7093 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:27.808028 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4f627d3f-d7f1-4035-b139-7cea922e7093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:27.839005 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ad2bc-922a-4c2b-9ea9-c425f2c944d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:27.840676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ad2bc-922a-4c2b-9ea9-c425f2c944d6 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:27.841277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ad2bc-922a-4c2b-9ea9-c425f2c944d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:27.841756 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ad2bc-922a-4c2b-9ea9-c425f2c944d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:27.842677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ad2bc-922a-4c2b-9ea9-c425f2c944d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:27.844544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ad2bc-922a-4c2b-9ea9-c425f2c944d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:27.846242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ad2bc-922a-4c2b-9ea9-c425f2c944d6 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:27.859109 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 73/156] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:27.862651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2038f9-e2c4-4642-b4f7-055b4f782861 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:27.863243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2038f9-e2c4-4642-b4f7-055b4f782861 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:27.863742 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2038f9-e2c4-4642-b4f7-055b4f782861 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:27.866992 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 74/157] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:27.868781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2038f9-e2c4-4642-b4f7-055b4f782861 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:27.931371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 84/158] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:27.933406 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f627d3f-d7f1-4035-b139-7cea922e7093 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:28.010309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:28.038630 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:28.092322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2038f9-e2c4-4642-b4f7-055b4f782861 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:28.118595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f627d3f-d7f1-4035-b139-7cea922e7093 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:28.168634 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96085d9-6b72-4852-94e2-9c1f624ef6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:28.170461 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96085d9-6b72-4852-94e2-9c1f624ef6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.171613 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96085d9-6b72-4852-94e2-9c1f624ef6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.174306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96085d9-6b72-4852-94e2-9c1f624ef6b7 None admin] PATH_INFO: `/v3/users/bb18b28bdf89484190c796b9fa8b60be` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.177397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96085d9-6b72-4852-94e2-9c1f624ef6b7 None admin] RBAC: Authorizing `identity:get_user(user_id=bb18b28bdf89484190c796b9fa8b60be)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:28.179313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96085d9-6b72-4852-94e2-9c1f624ef6b7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:28.190242 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 85/159] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:27 2022] GET /identity/v3/users/bb18b28bdf89484190c796b9fa8b60be => generated 289 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:28.191960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007762ef-35a2-4555-bc83-6f76211d9f8b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.195356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007762ef-35a2-4555-bc83-6f76211d9f8b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.195524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1c26f-6684-44bf-8caa-15c2b07ef846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:28.195524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1c26f-6684-44bf-8caa-15c2b07ef846 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.195524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1c26f-6684-44bf-8caa-15c2b07ef846 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.195524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1c26f-6684-44bf-8caa-15c2b07ef846 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.196840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007762ef-35a2-4555-bc83-6f76211d9f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.229699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1c26f-6684-44bf-8caa-15c2b07ef846 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:28.231449 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-007762ef-35a2-4555-bc83-6f76211d9f8b None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:28.238654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-007762ef-35a2-4555-bc83-6f76211d9f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-b9f1c26f-6684-44bf-8caa-15c2b07ef846 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:28.238847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:28.239695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:28.240569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:28.240569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:28.240569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:28.240569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:28.240569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:28.240569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:28.240569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:28.240569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:28.240569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:28.240569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:28.240569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:28.240569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:28.240569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 75/160] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:28.240569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1c26f-6684-44bf-8caa-15c2b07ef846 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:28.417532 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 86/161] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:28.417532 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007762ef-35a2-4555-bc83-6f76211d9f8b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:28.533404 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1c26f-6684-44bf-8caa-15c2b07ef846 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:28.535486 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:28.585465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:28.599080 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007762ef-35a2-4555-bc83-6f76211d9f8b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:28.653724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764544f-fcce-4732-8c38-0cea241284fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:28.654842 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764544f-fcce-4732-8c38-0cea241284fb None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.655339 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764544f-fcce-4732-8c38-0cea241284fb None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.655824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764544f-fcce-4732-8c38-0cea241284fb None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.665642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764544f-fcce-4732-8c38-0cea241284fb None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:28.668249 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764544f-fcce-4732-8c38-0cea241284fb None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:28.671519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 76/162] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:28 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:28.673632 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764544f-fcce-4732-8c38-0cea241284fb None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:28.674925 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18176ff6-4dac-43a1-ae82-dc6a042f9257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:28.675731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18176ff6-4dac-43a1-ae82-dc6a042f9257 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.676105 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18176ff6-4dac-43a1-ae82-dc6a042f9257 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.676388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18176ff6-4dac-43a1-ae82-dc6a042f9257 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.677121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18176ff6-4dac-43a1-ae82-dc6a042f9257 None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:28.677557 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18176ff6-4dac-43a1-ae82-dc6a042f9257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:28.679096 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18176ff6-4dac-43a1-ae82-dc6a042f9257 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:28.696633 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 87/163] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:28.697601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18176ff6-4dac-43a1-ae82-dc6a042f9257 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:28.900352 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18176ff6-4dac-43a1-ae82-dc6a042f9257 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:28.958394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:28.977088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bb5de-5f33-407d-be61-c0f3c29080e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:28.978381 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bb5de-5f33-407d-be61-c0f3c29080e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.979889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bb5de-5f33-407d-be61-c0f3c29080e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.981693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bb5de-5f33-407d-be61-c0f3c29080e0 None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf/users/bb18b28bdf89484190c796b9fa8b60be/roles/a915b705941d46538bd6a59aac287977` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.985227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bb5de-5f33-407d-be61-c0f3c29080e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=41be92ad8eba49128ba42ffcedab9caf, user_id=bb18b28bdf89484190c796b9fa8b60be, role_id=a915b705941d46538bd6a59aac287977)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:28.989061 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bb5de-5f33-407d-be61-c0f3c29080e0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:29.021229 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764544f-fcce-4732-8c38-0cea241284fb None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.064289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae00d6-f2d2-42a4-ac9f-092a8e3249cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.065324 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae00d6-f2d2-42a4-ac9f-092a8e3249cc None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.065820 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae00d6-f2d2-42a4-ac9f-092a8e3249cc None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.066312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae00d6-f2d2-42a4-ac9f-092a8e3249cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.080792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae00d6-f2d2-42a4-ac9f-092a8e3249cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-76ae00d6-f2d2-42a4-ac9f-092a8e3249cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:29.086734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:29.087767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:29.088588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:29.088588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:29.088588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:29.088588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:29.088588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:29.088588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:29.088588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:29.088588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:29.088588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:29.088588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:29.088588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:27:29.088588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:29.098469 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 88/164] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:28 2022] PUT /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf/users/bb18b28bdf89484190c796b9fa8b60be/roles/a915b705941d46538bd6a59aac287977 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:27:29.099760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bb5de-5f33-407d-be61-c0f3c29080e0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.106030 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 77/165] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:29.107306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae00d6-f2d2-42a4-ac9f-092a8e3249cc None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.236258 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:29.335928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae00d6-f2d2-42a4-ac9f-092a8e3249cc None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.402211 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bb5de-5f33-407d-be61-c0f3c29080e0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.406896 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d4632-59ea-4468-9ee9-b1311f68e5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.407821 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d4632-59ea-4468-9ee9-b1311f68e5bd None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.408307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d4632-59ea-4468-9ee9-b1311f68e5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.408765 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d4632-59ea-4468-9ee9-b1311f68e5bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.439735 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d4632-59ea-4468-9ee9-b1311f68e5bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-884d4632-59ea-4468-9ee9-b1311f68e5bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:29.445553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:29.446511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:29.447283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:29.447283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:29.447283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:29.447283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:29.447283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:29.447283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:29.447283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:29.447283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:29.447283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:29.447283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:29.447283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:29.447283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:29.456895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 78/166] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:29.458197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d4632-59ea-4468-9ee9-b1311f68e5bd None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.518779 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41785af-9625-4824-bde2-ec294d12cf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.519479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41785af-9625-4824-bde2-ec294d12cf52 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.519941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41785af-9625-4824-bde2-ec294d12cf52 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.520373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41785af-9625-4824-bde2-ec294d12cf52 None admin] PATH_INFO: `/v3/users/e0ff5ce14df94fc4a1f5f037539e037c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.524967 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41785af-9625-4824-bde2-ec294d12cf52 None admin] RBAC: Authorizing `identity:get_user(user_id=e0ff5ce14df94fc4a1f5f037539e037c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:29.527285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41785af-9625-4824-bde2-ec294d12cf52 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:29.534603 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 89/167] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:29 2022] GET /identity/v3/users/e0ff5ce14df94fc4a1f5f037539e037c => generated 275 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:29.534603 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41785af-9625-4824-bde2-ec294d12cf52 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.655407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d4632-59ea-4468-9ee9-b1311f68e5bd None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.661174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:29.676274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:29.700537 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a93c29-772f-49eb-b021-d7ba31a64202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.701493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a93c29-772f-49eb-b021-d7ba31a64202 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.701954 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a93c29-772f-49eb-b021-d7ba31a64202 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.702423 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a93c29-772f-49eb-b021-d7ba31a64202 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.703833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a93c29-772f-49eb-b021-d7ba31a64202 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:29.704313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a93c29-772f-49eb-b021-d7ba31a64202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:29.707577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a93c29-772f-49eb-b021-d7ba31a64202 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:29.729835 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 79/168] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:29.730815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a93c29-772f-49eb-b021-d7ba31a64202 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.756888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41785af-9625-4824-bde2-ec294d12cf52 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.838006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3dad0d-6d9c-487c-a7e7-8128700acf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.839055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3dad0d-6d9c-487c-a7e7-8128700acf45 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.839546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3dad0d-6d9c-487c-a7e7-8128700acf45 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.840011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3dad0d-6d9c-487c-a7e7-8128700acf45 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.840955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3dad0d-6d9c-487c-a7e7-8128700acf45 None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:29.841462 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3dad0d-6d9c-487c-a7e7-8128700acf45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:29.843309 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3dad0d-6d9c-487c-a7e7-8128700acf45 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:29.866163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 90/169] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:29.868628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3dad0d-6d9c-487c-a7e7-8128700acf45 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.907313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a93c29-772f-49eb-b021-d7ba31a64202 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.986089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889474c6-825b-46ee-9d22-cd8f7798c92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.986715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889474c6-825b-46ee-9d22-cd8f7798c92c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.986996 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889474c6-825b-46ee-9d22-cd8f7798c92c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.987269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889474c6-825b-46ee-9d22-cd8f7798c92c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.097170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:30.137981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889474c6-825b-46ee-9d22-cd8f7798c92c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:30.139551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889474c6-825b-46ee-9d22-cd8f7798c92c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:30.164294 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3dad0d-6d9c-487c-a7e7-8128700acf45 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:30.222255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:30.293197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0813f1f-eda2-42c4-a27f-a29fe7c7d549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:30.293869 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0813f1f-eda2-42c4-a27f-a29fe7c7d549 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:30.294145 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0813f1f-eda2-42c4-a27f-a29fe7c7d549 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:30.294413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0813f1f-eda2-42c4-a27f-a29fe7c7d549 None admin] PATH_INFO: `/v3/users/e0ff5ce14df94fc4a1f5f037539e037c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.295691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0813f1f-eda2-42c4-a27f-a29fe7c7d549 None admin] RBAC: Authorizing `identity:get_user(user_id=e0ff5ce14df94fc4a1f5f037539e037c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:30.297643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0813f1f-eda2-42c4-a27f-a29fe7c7d549 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:30.300358 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 91/170] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:29 2022] GET /identity/v3/users/e0ff5ce14df94fc4a1f5f037539e037c => generated 275 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:30.301661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0813f1f-eda2-42c4-a27f-a29fe7c7d549 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:30.328926 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 80/171] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:30.330755 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaeb8c6-b1f2-4761-a0f3-51d04c075c12 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:30.331052 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaeb8c6-b1f2-4761-a0f3-51d04c075c12 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:30.331452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaeb8c6-b1f2-4761-a0f3-51d04c075c12 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.332877 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 81/172] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:30.333701 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaeb8c6-b1f2-4761-a0f3-51d04c075c12 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:30.540828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:30.557607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0813f1f-eda2-42c4-a27f-a29fe7c7d549 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:30.602150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaeb8c6-b1f2-4761-a0f3-51d04c075c12 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:30.603652 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6d753-3b6b-41c9-869b-5fc0355a41b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:30.604307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6d753-3b6b-41c9-869b-5fc0355a41b9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:30.604627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6d753-3b6b-41c9-869b-5fc0355a41b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:30.604902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6d753-3b6b-41c9-869b-5fc0355a41b9 None admin] PATH_INFO: `/v3/users/e0ff5ce14df94fc4a1f5f037539e037c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.606137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6d753-3b6b-41c9-869b-5fc0355a41b9 None admin] RBAC: Authorizing `identity:get_user(user_id=e0ff5ce14df94fc4a1f5f037539e037c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:30.607557 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6d753-3b6b-41c9-869b-5fc0355a41b9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:30.610143 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 92/173] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:30 2022] GET /identity/v3/users/e0ff5ce14df94fc4a1f5f037539e037c => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:30.610996 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6d753-3b6b-41c9-869b-5fc0355a41b9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:30.686179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133103cd-d320-4def-8c3c-1fd3678ea88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:30.686816 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133103cd-d320-4def-8c3c-1fd3678ea88c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:30.687099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133103cd-d320-4def-8c3c-1fd3678ea88c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:30.687375 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133103cd-d320-4def-8c3c-1fd3678ea88c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.713181 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:30.793483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6d753-3b6b-41c9-869b-5fc0355a41b9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:30.863066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133103cd-d320-4def-8c3c-1fd3678ea88c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:30.865630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133103cd-d320-4def-8c3c-1fd3678ea88c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:30.869317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbe496-a8cd-4257-91b8-641a58b8b6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:30.871799 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbe496-a8cd-4257-91b8-641a58b8b6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:30.872302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbe496-a8cd-4257-91b8-641a58b8b6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:30.872853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbe496-a8cd-4257-91b8-641a58b8b6a7 None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.875009 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dbe496-a8cd-4257-91b8-641a58b8b6a7 None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:30.878753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dbe496-a8cd-4257-91b8-641a58b8b6a7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:30.892154 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 93/174] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:30 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:30.892154 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50543f2f-06bc-4690-955f-4c66cb9696db None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:30.892154 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50543f2f-06bc-4690-955f-4c66cb9696db None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:30.892154 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50543f2f-06bc-4690-955f-4c66cb9696db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.927340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-50543f2f-06bc-4690-955f-4c66cb9696db None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:30.930251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-50543f2f-06bc-4690-955f-4c66cb9696db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:31.024645 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:31.057776 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 94/175] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:31.064436 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50543f2f-06bc-4690-955f-4c66cb9696db None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:31.103642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 82/176] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 770 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:31.112437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2746f1-9860-4a22-9283-0b25e67299d5 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.113612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2746f1-9860-4a22-9283-0b25e67299d5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.114603 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2746f1-9860-4a22-9283-0b25e67299d5 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.117556 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 83/177] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:31 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:31.119312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2746f1-9860-4a22-9283-0b25e67299d5 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:31.191325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:31.270011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50543f2f-06bc-4690-955f-4c66cb9696db None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:31.298911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2746f1-9860-4a22-9283-0b25e67299d5 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:31.313959 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ca6c6-ba7e-411f-9c21-34667abd8462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:31.314505 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ca6c6-ba7e-411f-9c21-34667abd8462 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.314778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ca6c6-ba7e-411f-9c21-34667abd8462 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.315056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ca6c6-ba7e-411f-9c21-34667abd8462 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.343900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268d5b7-858a-4eff-8ffc-6cf338c25f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:31.345035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268d5b7-858a-4eff-8ffc-6cf338c25f8b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.345326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268d5b7-858a-4eff-8ffc-6cf338c25f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.345584 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268d5b7-858a-4eff-8ffc-6cf338c25f8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.346320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268d5b7-858a-4eff-8ffc-6cf338c25f8b None admin] RBAC: Adding query filter params (role.id=c7b826398b4e466491d4723321aaa5f1, scope.project.id=41be92ad8eba49128ba42ffcedab9caf, user.id=e0ff5ce14df94fc4a1f5f037539e037c) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:31.346623 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268d5b7-858a-4eff-8ffc-6cf338c25f8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:31.348085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268d5b7-858a-4eff-8ffc-6cf338c25f8b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:31.354988 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 84/178] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:31 2022] GET /identity/v3/role_assignments?user.id=e0ff5ce14df94fc4a1f5f037539e037c&scope.project.id=41be92ad8eba49128ba42ffcedab9caf&role.id=c7b826398b4e466491d4723321aaa5f1 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:31.355874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268d5b7-858a-4eff-8ffc-6cf338c25f8b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:31.409650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ca6c6-ba7e-411f-9c21-34667abd8462 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:31.412318 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ca6c6-ba7e-411f-9c21-34667abd8462 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:31.420558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:31.496810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:31.506660 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268d5b7-858a-4eff-8ffc-6cf338c25f8b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:31.558812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f908b-a3a2-4d81-a5dd-423dc6481352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:31.559715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f908b-a3a2-4d81-a5dd-423dc6481352 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.560139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f908b-a3a2-4d81-a5dd-423dc6481352 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.560600 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f908b-a3a2-4d81-a5dd-423dc6481352 None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.563000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f908b-a3a2-4d81-a5dd-423dc6481352 None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:31.564935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f908b-a3a2-4d81-a5dd-423dc6481352 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:31.566915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 85/179] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:31 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:31.568554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd750b8-8f96-4568-99f0-70cdb1b09e98 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.568995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd750b8-8f96-4568-99f0-70cdb1b09e98 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.569436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd750b8-8f96-4568-99f0-70cdb1b09e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.569892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 95/180] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:31.571107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056cebd-8965-4774-897f-347756f16bf2 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.571454 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056cebd-8965-4774-897f-347756f16bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.571882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056cebd-8965-4774-897f-347756f16bf2 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.576197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 96/181] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:31.576197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056cebd-8965-4774-897f-347756f16bf2 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:31.597511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-bcd750b8-8f96-4568-99f0-70cdb1b09e98 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:31.599061 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-bcd750b8-8f96-4568-99f0-70cdb1b09e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:31.704690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 86/182] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:31.708620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd750b8-8f96-4568-99f0-70cdb1b09e98 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:31.721306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056cebd-8965-4774-897f-347756f16bf2 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:31.759490 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:31.771786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbeea38-0646-4bdd-9a51-1eaf56cb9756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:31.773141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbeea38-0646-4bdd-9a51-1eaf56cb9756 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.773619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbeea38-0646-4bdd-9a51-1eaf56cb9756 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.774286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbeea38-0646-4bdd-9a51-1eaf56cb9756 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.791188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbeea38-0646-4bdd-9a51-1eaf56cb9756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-9cbeea38-0646-4bdd-9a51-1eaf56cb9756 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:31.798185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:31.799139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:31.799922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:31.799922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:31.799922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:31.799922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:31.799922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:31.799922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:31.799922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:31.799922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:31.799922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:31.799922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:31.799922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:31.799922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:31.802181 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 97/183] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:31.807627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbeea38-0646-4bdd-9a51-1eaf56cb9756 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:31.828118 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:31.841734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd750b8-8f96-4568-99f0-70cdb1b09e98 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:31.885231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35dbf6b-c7c2-4b69-96fb-3a11c3589287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:31.885849 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35dbf6b-c7c2-4b69-96fb-3a11c3589287 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.886134 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35dbf6b-c7c2-4b69-96fb-3a11c3589287 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.886411 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35dbf6b-c7c2-4b69-96fb-3a11c3589287 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.887157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35dbf6b-c7c2-4b69-96fb-3a11c3589287 None admin] RBAC: Adding query filter params (role.id=003cfd128ecf4c238197991ad36f0c9a, scope.project.id=41be92ad8eba49128ba42ffcedab9caf, user.id=e0ff5ce14df94fc4a1f5f037539e037c) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:31.887537 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35dbf6b-c7c2-4b69-96fb-3a11c3589287 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:31.889058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35dbf6b-c7c2-4b69-96fb-3a11c3589287 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:31.896429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 87/184] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:31 2022] GET /identity/v3/role_assignments?user.id=e0ff5ce14df94fc4a1f5f037539e037c&scope.project.id=41be92ad8eba49128ba42ffcedab9caf&role.id=003cfd128ecf4c238197991ad36f0c9a => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:31.897888 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cc924-cdc9-46c7-855a-58c672028811 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.898180 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cc924-cdc9-46c7-855a-58c672028811 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.898454 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cc924-cdc9-46c7-855a-58c672028811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.919073 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-bd1cc924-cdc9-46c7-855a-58c672028811 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:31.920291 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-bd1cc924-cdc9-46c7-855a-58c672028811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:31.948631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbeea38-0646-4bdd-9a51-1eaf56cb9756 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.012611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 88/185] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:32.013318 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cc924-cdc9-46c7-855a-58c672028811 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.013647 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bfc1a-5eed-4892-8db7-0ac17356dd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.014401 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bfc1a-5eed-4892-8db7-0ac17356dd08 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.014754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bfc1a-5eed-4892-8db7-0ac17356dd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.015102 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bfc1a-5eed-4892-8db7-0ac17356dd08 None admin] PATH_INFO: `/v3/projects/db01eabb387846ca826fb4ef90031208` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.018177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bfc1a-5eed-4892-8db7-0ac17356dd08 None admin] RBAC: Authorizing `identity:get_project(project_id=db01eabb387846ca826fb4ef90031208)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.019918 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bfc1a-5eed-4892-8db7-0ac17356dd08 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.022047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 98/186] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:31 2022] GET /identity/v3/projects/db01eabb387846ca826fb4ef90031208 => generated 312 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:32.024250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809a98d-a22f-459e-a8bb-92722b29db37 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.024635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809a98d-a22f-459e-a8bb-92722b29db37 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.024983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809a98d-a22f-459e-a8bb-92722b29db37 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.026935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 99/187] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:32.032389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809a98d-a22f-459e-a8bb-92722b29db37 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.058302 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:32.133114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:32.136435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cc924-cdc9-46c7-855a-58c672028811 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.156265 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809a98d-a22f-459e-a8bb-92722b29db37 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.198569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1bb78-c045-46ca-9af8-126cf6c1d2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.199646 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1bb78-c045-46ca-9af8-126cf6c1d2df None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.199990 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1bb78-c045-46ca-9af8-126cf6c1d2df None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.202920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1bb78-c045-46ca-9af8-126cf6c1d2df None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.202920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1bb78-c045-46ca-9af8-126cf6c1d2df None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:32.202920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1bb78-c045-46ca-9af8-126cf6c1d2df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.203267 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8689f1-e015-4f48-82b0-e7f81df25521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.203359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1bb78-c045-46ca-9af8-126cf6c1d2df None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.203428 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8689f1-e015-4f48-82b0-e7f81df25521 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.203921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8689f1-e015-4f48-82b0-e7f81df25521 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.204475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8689f1-e015-4f48-82b0-e7f81df25521 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.206099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8689f1-e015-4f48-82b0-e7f81df25521 None admin] RBAC: Adding query filter params (role.id=c7b826398b4e466491d4723321aaa5f1, scope.project.id=db01eabb387846ca826fb4ef90031208, user.id=e0ff5ce14df94fc4a1f5f037539e037c) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:32.206538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8689f1-e015-4f48-82b0-e7f81df25521 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.209333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8689f1-e015-4f48-82b0-e7f81df25521 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.214586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 89/188] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:32.216586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1371f-8277-4bf4-915b-64b90920e2ce None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.216876 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1371f-8277-4bf4-915b-64b90920e2ce None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.217152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1371f-8277-4bf4-915b-64b90920e2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.222215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 100/189] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/role_assignments?user.id=e0ff5ce14df94fc4a1f5f037539e037c&scope.project.id=db01eabb387846ca826fb4ef90031208&role.id=c7b826398b4e466491d4723321aaa5f1 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:32.224018 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8689f1-e015-4f48-82b0-e7f81df25521 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.257494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-c9f1371f-8277-4bf4-915b-64b90920e2ce None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:32.258790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c9f1371f-8277-4bf4-915b-64b90920e2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:32.368915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 90/190] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:32.371430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1371f-8277-4bf4-915b-64b90920e2ce None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.380434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8689f1-e015-4f48-82b0-e7f81df25521 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.420524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:32.434317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdb677-5640-4e58-bfad-2158af042405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.434965 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdb677-5640-4e58-bfad-2158af042405 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.435243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdb677-5640-4e58-bfad-2158af042405 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.435513 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdb677-5640-4e58-bfad-2158af042405 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.436748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdb677-5640-4e58-bfad-2158af042405 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.438247 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdb677-5640-4e58-bfad-2158af042405 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.440116 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 101/191] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:32.441394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdb677-5640-4e58-bfad-2158af042405 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.469408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:32.565780 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1371f-8277-4bf4-915b-64b90920e2ce None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.589826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdb677-5640-4e58-bfad-2158af042405 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.611325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7a4d1-807b-4cb2-bb79-e425f4e4b64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.611992 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7a4d1-807b-4cb2-bb79-e425f4e4b64a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.612270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7a4d1-807b-4cb2-bb79-e425f4e4b64a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.612579 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7a4d1-807b-4cb2-bb79-e425f4e4b64a None admin] PATH_INFO: `/v3/users/4ecfa39ed2b44f759641959889c4eb9b` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.629682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7a4d1-807b-4cb2-bb79-e425f4e4b64a None admin] RBAC: Authorizing `identity:get_user(user_id=4ecfa39ed2b44f759641959889c4eb9b)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.631208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7a4d1-807b-4cb2-bb79-e425f4e4b64a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.634046 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 91/192] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/users/4ecfa39ed2b44f759641959889c4eb9b => generated 283 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:32.635407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7a4d1-807b-4cb2-bb79-e425f4e4b64a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.639781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10527d9c-2896-410b-bc59-65d1f91432de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.640759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10527d9c-2896-410b-bc59-65d1f91432de None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.641203 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10527d9c-2896-410b-bc59-65d1f91432de None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.649213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10527d9c-2896-410b-bc59-65d1f91432de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.658882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10527d9c-2896-410b-bc59-65d1f91432de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-10527d9c-2896-410b-bc59-65d1f91432de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:32.663762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:32.665279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:32.666934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:32.666934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:32.666934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:32.666934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:32.666934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:32.666934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:32.666934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:32.666934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:32.666934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:32.666934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:32.666934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:27:32.666934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:32.666934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 102/193] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:32.666934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10527d9c-2896-410b-bc59-65d1f91432de None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.757920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:32.759178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:32.778513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7a4d1-807b-4cb2-bb79-e425f4e4b64a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.791064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10527d9c-2896-410b-bc59-65d1f91432de None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.818467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b57fd3-9135-4e40-9beb-9dd0315e0ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.819089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b57fd3-9135-4e40-9beb-9dd0315e0ae5 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.819431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b57fd3-9135-4e40-9beb-9dd0315e0ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.819705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b57fd3-9135-4e40-9beb-9dd0315e0ae5 None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.820382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b57fd3-9135-4e40-9beb-9dd0315e0ae5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.821980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b57fd3-9135-4e40-9beb-9dd0315e0ae5 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.828710 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c11a1a-33cf-4495-b29e-f5a6abfdb8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.829317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c11a1a-33cf-4495-b29e-f5a6abfdb8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.829592 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c11a1a-33cf-4495-b29e-f5a6abfdb8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.829857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c11a1a-33cf-4495-b29e-f5a6abfdb8d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.898357 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c11a1a-33cf-4495-b29e-f5a6abfdb8d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.899844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c11a1a-33cf-4495-b29e-f5a6abfdb8d0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.944372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 92/194] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:27:32 2022] POST /identity/v3/users => generated 297 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:32.945395 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b57fd3-9135-4e40-9beb-9dd0315e0ae5 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.999442 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:33.046978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 103/195] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:33.046978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c11a1a-33cf-4495-b29e-f5a6abfdb8d0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:33.093587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b57fd3-9135-4e40-9beb-9dd0315e0ae5 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:33.138770 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a08904-8bad-41cb-9713-29be332706cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:33.139518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a08904-8bad-41cb-9713-29be332706cd None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:33.139876 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a08904-8bad-41cb-9713-29be332706cd None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:33.140222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a08904-8bad-41cb-9713-29be332706cd None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:33.143020 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a08904-8bad-41cb-9713-29be332706cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:33.143020 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a08904-8bad-41cb-9713-29be332706cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:33.143567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a08904-8bad-41cb-9713-29be332706cd None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:33.146225 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:33.154674 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 93/196] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:33.154674 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a08904-8bad-41cb-9713-29be332706cd None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:33.197305 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c11a1a-33cf-4495-b29e-f5a6abfdb8d0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:33.250335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06126a54-ba41-48bb-a088-b0ded5790d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:33.251353 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06126a54-ba41-48bb-a088-b0ded5790d3e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:33.251832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06126a54-ba41-48bb-a088-b0ded5790d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:33.252519 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06126a54-ba41-48bb-a088-b0ded5790d3e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:33.276656 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06126a54-ba41-48bb-a088-b0ded5790d3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-06126a54-ba41-48bb-a088-b0ded5790d3e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:33.282538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:33.284339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:33.285913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:33.285913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:33.285913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:33.285913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:33.285913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:33.285913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:33.285913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:33.285913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:33.285913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:33.285913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:33.285913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:33.285913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:33.285913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 104/197] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:33.285913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06126a54-ba41-48bb-a088-b0ded5790d3e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:33.330238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a08904-8bad-41cb-9713-29be332706cd None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:33.340212 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:33.385681 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d488e-0ac3-4a65-9bcc-caeeddc09726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:33.386467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d488e-0ac3-4a65-9bcc-caeeddc09726 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:33.386597 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d488e-0ac3-4a65-9bcc-caeeddc09726 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:33.386966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d488e-0ac3-4a65-9bcc-caeeddc09726 None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:33.388003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d488e-0ac3-4a65-9bcc-caeeddc09726 None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:33.390003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d488e-0ac3-4a65-9bcc-caeeddc09726 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:33.391806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 94/198] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:33 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:33.392475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d488e-0ac3-4a65-9bcc-caeeddc09726 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:33.435640 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06126a54-ba41-48bb-a088-b0ded5790d3e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:33.473665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9911b-dbaa-4f61-ad7c-aa00900b6ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:33.474338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9911b-dbaa-4f61-ad7c-aa00900b6ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:33.474622 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9911b-dbaa-4f61-ad7c-aa00900b6ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:33.474900 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9911b-dbaa-4f61-ad7c-aa00900b6ed7 None admin] PATH_INFO: `/v3/users/4ecfa39ed2b44f759641959889c4eb9b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:33.476187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9911b-dbaa-4f61-ad7c-aa00900b6ed7 None admin] RBAC: Authorizing `identity:get_user(user_id=4ecfa39ed2b44f759641959889c4eb9b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:33.477843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9911b-dbaa-4f61-ad7c-aa00900b6ed7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:33.479547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 105/199] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:33 2022] GET /identity/v3/users/4ecfa39ed2b44f759641959889c4eb9b => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:33.481572 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:33.485184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9911b-dbaa-4f61-ad7c-aa00900b6ed7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:33.526069 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d488e-0ac3-4a65-9bcc-caeeddc09726 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:33.577026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8f285-7f21-49ba-b555-41161af8b01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:33.577921 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8f285-7f21-49ba-b555-41161af8b01a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:33.578895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8f285-7f21-49ba-b555-41161af8b01a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:33.579270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8f285-7f21-49ba-b555-41161af8b01a None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:33.580136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8f285-7f21-49ba-b555-41161af8b01a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:33.580696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8f285-7f21-49ba-b555-41161af8b01a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:33.582485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8f285-7f21-49ba-b555-41161af8b01a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:33.593167 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 95/200] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:33.594781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8f285-7f21-49ba-b555-41161af8b01a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:33.601550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:33.604628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9911b-dbaa-4f61-ad7c-aa00900b6ed7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:33.643357 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ef1f8-10d6-465b-9c39-53f38aea8549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:33.644154 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ef1f8-10d6-465b-9c39-53f38aea8549 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:33.644620 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ef1f8-10d6-465b-9c39-53f38aea8549 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:33.644997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ef1f8-10d6-465b-9c39-53f38aea8549 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:33.645864 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ef1f8-10d6-465b-9c39-53f38aea8549 None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15, user.id=4ecfa39ed2b44f759641959889c4eb9b) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:33.647378 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ef1f8-10d6-465b-9c39-53f38aea8549 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:33.649042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ef1f8-10d6-465b-9c39-53f38aea8549 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:33.655491 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 106/201] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:33 2022] GET /identity/v3/role_assignments?user.id=4ecfa39ed2b44f759641959889c4eb9b&scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15&role.id=e7f48343fc1542979ab4459865109e30 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:33.657140 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ef1f8-10d6-465b-9c39-53f38aea8549 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:33.735769 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8f285-7f21-49ba-b555-41161af8b01a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:33.812833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ef1f8-10d6-465b-9c39-53f38aea8549 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:33.823326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:33.848462 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c4211-f517-49a3-bd65-2d94cf10a2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:33.849047 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c4211-f517-49a3-bd65-2d94cf10a2fb None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:33.849332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c4211-f517-49a3-bd65-2d94cf10a2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:33.849611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c4211-f517-49a3-bd65-2d94cf10a2fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:33.888513 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a09dc-b2e7-4e4b-a576-d54c04eb7af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:33.890409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a09dc-b2e7-4e4b-a576-d54c04eb7af9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:33.892160 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a09dc-b2e7-4e4b-a576-d54c04eb7af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:33.892708 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a09dc-b2e7-4e4b-a576-d54c04eb7af9 None admin] PATH_INFO: `/v3/users/bb18b28bdf89484190c796b9fa8b60be` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:33.894608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a09dc-b2e7-4e4b-a576-d54c04eb7af9 None admin] RBAC: Authorizing `identity:get_user(user_id=bb18b28bdf89484190c796b9fa8b60be)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:33.897522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a09dc-b2e7-4e4b-a576-d54c04eb7af9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:33.899788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 107/202] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:33 2022] GET /identity/v3/users/bb18b28bdf89484190c796b9fa8b60be => generated 289 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:33.906326 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a09dc-b2e7-4e4b-a576-d54c04eb7af9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:33.961009 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c4211-f517-49a3-bd65-2d94cf10a2fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:33.962517 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c4211-f517-49a3-bd65-2d94cf10a2fb None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:33.981164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:34.183809 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a09dc-b2e7-4e4b-a576-d54c04eb7af9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:34.245446 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 96/203] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:34.252488 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c4211-f517-49a3-bd65-2d94cf10a2fb None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:34.300541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55bf7e-e9f0-4d6c-93b8-c653426f4762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:34.301570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55bf7e-e9f0-4d6c-93b8-c653426f4762 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:34.302301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55bf7e-e9f0-4d6c-93b8-c653426f4762 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:34.302791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55bf7e-e9f0-4d6c-93b8-c653426f4762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:34.322307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:34.459644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c4211-f517-49a3-bd65-2d94cf10a2fb None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:34.489493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55bf7e-e9f0-4d6c-93b8-c653426f4762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:34.491364 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55bf7e-e9f0-4d6c-93b8-c653426f4762 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:34.500430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5a140-a053-48cc-ad1e-82fb6f627875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:34.500430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5a140-a053-48cc-ad1e-82fb6f627875 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:34.500430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5a140-a053-48cc-ad1e-82fb6f627875 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:34.500430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5a140-a053-48cc-ad1e-82fb6f627875 None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:34.501812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5a140-a053-48cc-ad1e-82fb6f627875 None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:34.504705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5a140-a053-48cc-ad1e-82fb6f627875 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:34.512514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 97/204] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:34 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:34.517438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5a140-a053-48cc-ad1e-82fb6f627875 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:34.563591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:34.696554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 108/205] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 791 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:34.703931 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55bf7e-e9f0-4d6c-93b8-c653426f4762 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:34.719106 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:34.728928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5a140-a053-48cc-ad1e-82fb6f627875 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:34.804269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8390fb-3688-4b0d-b73d-f6f220a16fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:34.804928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8390fb-3688-4b0d-b73d-f6f220a16fcf None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:34.805209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8390fb-3688-4b0d-b73d-f6f220a16fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:34.805482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8390fb-3688-4b0d-b73d-f6f220a16fcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:34.806232 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8390fb-3688-4b0d-b73d-f6f220a16fcf None admin] RBAC: Adding query filter params (role.id=003cfd128ecf4c238197991ad36f0c9a, scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15, user.id=4ecfa39ed2b44f759641959889c4eb9b) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:34.806543 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8390fb-3688-4b0d-b73d-f6f220a16fcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:34.808312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8390fb-3688-4b0d-b73d-f6f220a16fcf None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:34.822310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 98/206] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:34 2022] GET /identity/v3/role_assignments?user.id=4ecfa39ed2b44f759641959889c4eb9b&scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15&role.id=003cfd128ecf4c238197991ad36f0c9a => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:34.927260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55bf7e-e9f0-4d6c-93b8-c653426f4762 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:35.007929 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:35.018286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cb77e-d4b8-4300-94a3-0727c0f34731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:35.019259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cb77e-d4b8-4300-94a3-0727c0f34731 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.019681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cb77e-d4b8-4300-94a3-0727c0f34731 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.020119 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cb77e-d4b8-4300-94a3-0727c0f34731 None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.026604 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cb77e-d4b8-4300-94a3-0727c0f34731 None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:35.026604 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cb77e-d4b8-4300-94a3-0727c0f34731 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:35.027829 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 109/207] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:34 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:35.033805 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8390fb-3688-4b0d-b73d-f6f220a16fcf None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:35.273935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8390fb-3688-4b0d-b73d-f6f220a16fcf None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:35.334356 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bbbc6-bd9c-4e78-bbad-1072cb6f326d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:35.335013 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bbbc6-bd9c-4e78-bbad-1072cb6f326d None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.335326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bbbc6-bd9c-4e78-bbad-1072cb6f326d None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.335631 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bbbc6-bd9c-4e78-bbad-1072cb6f326d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.336452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bbbc6-bd9c-4e78-bbad-1072cb6f326d None admin] RBAC: Adding query filter params (role.id=a915b705941d46538bd6a59aac287977, scope.project.id=41be92ad8eba49128ba42ffcedab9caf, user.id=bb18b28bdf89484190c796b9fa8b60be) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:35.336783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bbbc6-bd9c-4e78-bbad-1072cb6f326d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:35.338297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bbbc6-bd9c-4e78-bbad-1072cb6f326d None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:35.348105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 99/208] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:35 2022] GET /identity/v3/role_assignments?user.id=bb18b28bdf89484190c796b9fa8b60be&scope.project.id=41be92ad8eba49128ba42ffcedab9caf&role.id=a915b705941d46538bd6a59aac287977 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:35.595142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec130e-ff99-41d6-927e-1d66bce6fb77 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.595483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec130e-ff99-41d6-927e-1d66bce6fb77 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.595715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec130e-ff99-41d6-927e-1d66bce6fb77 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.597469 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 110/209] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:35.603654 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb4cfa-fecf-4989-83f6-a10e71c5ed2a None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.604154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb4cfa-fecf-4989-83f6-a10e71c5ed2a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.604378 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb4cfa-fecf-4989-83f6-a10e71c5ed2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.628136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-e5bb4cfa-fecf-4989-83f6-a10e71c5ed2a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:35.629453 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e5bb4cfa-fecf-4989-83f6-a10e71c5ed2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:35.649997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:35.707531 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 100/210] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:35.765275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9a452-e35f-4a2f-999e-f66d6410fb00 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.766311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9a452-e35f-4a2f-999e-f66d6410fb00 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.766384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9a452-e35f-4a2f-999e-f66d6410fb00 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.768287 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 111/211] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:35.774248 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385d586-eb93-4d6d-a7a1-5ee5468746d5 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.774680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385d586-eb93-4d6d-a7a1-5ee5468746d5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.775008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385d586-eb93-4d6d-a7a1-5ee5468746d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.795734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-d385d586-eb93-4d6d-a7a1-5ee5468746d5 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:35.796932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d385d586-eb93-4d6d-a7a1-5ee5468746d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:35.878892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 101/212] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:36.026187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbd501-6cb7-43ad-848a-466b09962b5a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:36.026537 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbd501-6cb7-43ad-848a-466b09962b5a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:36.026814 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbd501-6cb7-43ad-848a-466b09962b5a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:36.028383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 112/213] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:36.034895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5e790-ce22-4dab-9fdc-ac5148317fa6 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:36.035187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5e790-ce22-4dab-9fdc-ac5148317fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:36.035455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5e790-ce22-4dab-9fdc-ac5148317fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:36.052032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbd501-6cb7-43ad-848a-466b09962b5a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:36.058264 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-8ca5e790-ce22-4dab-9fdc-ac5148317fa6 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:36.060133 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-8ca5e790-ce22-4dab-9fdc-ac5148317fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:36.139302 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 102/214] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:36.171996 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5e790-ce22-4dab-9fdc-ac5148317fa6 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:36.176275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbd501-6cb7-43ad-848a-466b09962b5a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:36.189596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:36.223515 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016aa50-8f16-4047-b845-b350218bf771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:36.225099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016aa50-8f16-4047-b845-b350218bf771 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:36.226835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016aa50-8f16-4047-b845-b350218bf771 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:36.227548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016aa50-8f16-4047-b845-b350218bf771 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:36.229590 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016aa50-8f16-4047-b845-b350218bf771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:36.232011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016aa50-8f16-4047-b845-b350218bf771 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:36.235089 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 113/215] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:36.242213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016aa50-8f16-4047-b845-b350218bf771 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:36.304290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5e790-ce22-4dab-9fdc-ac5148317fa6 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:36.333787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:36.346267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184780a6-f04e-4a33-94d8-e0d5e30e6231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:36.347242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184780a6-f04e-4a33-94d8-e0d5e30e6231 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:36.347768 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184780a6-f04e-4a33-94d8-e0d5e30e6231 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:36.348975 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184780a6-f04e-4a33-94d8-e0d5e30e6231 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:36.361651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184780a6-f04e-4a33-94d8-e0d5e30e6231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-184780a6-f04e-4a33-94d8-e0d5e30e6231 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:36.367829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:36.368824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:36.369612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:36.369612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:36.369612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:36.369612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:36.369612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:36.369612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:36.369612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:36.369612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:36.369612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:36.369612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:36.369612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:36.369612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:36.372218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 103/216] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:36.379131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184780a6-f04e-4a33-94d8-e0d5e30e6231 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:36.380652 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016aa50-8f16-4047-b845-b350218bf771 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:36.420528 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7145a-0dc0-4e89-856d-f63e5b1df832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:36.421272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7145a-0dc0-4e89-856d-f63e5b1df832 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:36.421459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7145a-0dc0-4e89-856d-f63e5b1df832 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:36.421728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7145a-0dc0-4e89-856d-f63e5b1df832 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:36.422399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7145a-0dc0-4e89-856d-f63e5b1df832 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:36.423895 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7145a-0dc0-4e89-856d-f63e5b1df832 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:36.475484 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:36.504843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184780a6-f04e-4a33-94d8-e0d5e30e6231 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:36.540138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb33ab-eb7a-4877-adb5-a6434be5be90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:36.540797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb33ab-eb7a-4877-adb5-a6434be5be90 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:36.541073 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb33ab-eb7a-4877-adb5-a6434be5be90 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:36.541357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb33ab-eb7a-4877-adb5-a6434be5be90 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:36.542362 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb33ab-eb7a-4877-adb5-a6434be5be90 None admin] RBAC: Adding query filter params (name=member) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:36.542362 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb33ab-eb7a-4877-adb5-a6434be5be90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:36.543783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb33ab-eb7a-4877-adb5-a6434be5be90 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:36.553723 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 104/217] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:36.553723 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb33ab-eb7a-4877-adb5-a6434be5be90 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:36.561372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 114/218] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:27:36 2022] POST /identity/v3/users => generated 297 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:27:36.563128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7145a-0dc0-4e89-856d-f63e5b1df832 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:36.694684 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7145a-0dc0-4e89-856d-f63e5b1df832 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:36.717296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb33ab-eb7a-4877-adb5-a6434be5be90 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:36.725771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:36.739408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a33ce1-09a8-477f-8ccd-74eb534d9f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:36.740080 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a33ce1-09a8-477f-8ccd-74eb534d9f1a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:36.740470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a33ce1-09a8-477f-8ccd-74eb534d9f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:36.740710 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a33ce1-09a8-477f-8ccd-74eb534d9f1a None admin] PATH_INFO: `/v3/users/0b0ae20c19bf407cb727570d3f18b893` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:36.758857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a33ce1-09a8-477f-8ccd-74eb534d9f1a None admin] RBAC: Authorizing `identity:get_user(user_id=0b0ae20c19bf407cb727570d3f18b893)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:36.760466 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a33ce1-09a8-477f-8ccd-74eb534d9f1a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:36.762502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 115/219] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:36 2022] GET /identity/v3/users/0b0ae20c19bf407cb727570d3f18b893 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:36.768558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a33ce1-09a8-477f-8ccd-74eb534d9f1a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:36.794469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccfe0c-cc71-49b7-bd42-ad35ed6acb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:36.795606 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccfe0c-cc71-49b7-bd42-ad35ed6acb8b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:36.796093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccfe0c-cc71-49b7-bd42-ad35ed6acb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:36.797302 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccfe0c-cc71-49b7-bd42-ad35ed6acb8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:36.803301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:36.818814 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccfe0c-cc71-49b7-bd42-ad35ed6acb8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-dcccfe0c-cc71-49b7-bd42-ad35ed6acb8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:36.822998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:36.824085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:36.824963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:36.824963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:36.824963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:36.824963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:36.824963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:36.824963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:36.824963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:36.824963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:36.824963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:36.824963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:36.824963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:36.824963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:36.824963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 105/220] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:36.835669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccfe0c-cc71-49b7-bd42-ad35ed6acb8b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:36.945159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a33ce1-09a8-477f-8ccd-74eb534d9f1a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:36.987915 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8be3bb-d98a-4603-9ecc-609113996a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:36.988961 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8be3bb-d98a-4603-9ecc-609113996a6c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:36.989495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8be3bb-d98a-4603-9ecc-609113996a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:36.989970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8be3bb-d98a-4603-9ecc-609113996a6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.027259 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccfe0c-cc71-49b7-bd42-ad35ed6acb8b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:37.088948 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:37.091011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d4f8f-dba3-4e80-86f6-908584e40f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:37.091749 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d4f8f-dba3-4e80-86f6-908584e40f0e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.092009 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d4f8f-dba3-4e80-86f6-908584e40f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.092283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d4f8f-dba3-4e80-86f6-908584e40f0e None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.093059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2d4f8f-dba3-4e80-86f6-908584e40f0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:37.093368 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2d4f8f-dba3-4e80-86f6-908584e40f0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:37.095364 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2d4f8f-dba3-4e80-86f6-908584e40f0e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:37.105243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8be3bb-d98a-4603-9ecc-609113996a6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:37.108460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8be3bb-d98a-4603-9ecc-609113996a6c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:37.111981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 106/221] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:37.117521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d4f8f-dba3-4e80-86f6-908584e40f0e None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:37.293965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:37.370928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d4f8f-dba3-4e80-86f6-908584e40f0e None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:37.409542 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6bc18d-3b39-445f-bc28-917472bc6353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:37.410187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 116/222] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:27:37.410295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bc18d-3b39-445f-bc28-917472bc6353 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.410534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bc18d-3b39-445f-bc28-917472bc6353 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.410884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bc18d-3b39-445f-bc28-917472bc6353 None admin] PATH_INFO: `/v3/users/4ecfa39ed2b44f759641959889c4eb9b` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.413343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6bc18d-3b39-445f-bc28-917472bc6353 None admin] RBAC: Authorizing `identity:get_user(user_id=4ecfa39ed2b44f759641959889c4eb9b)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:37.415407 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8be3bb-d98a-4603-9ecc-609113996a6c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:37.415530 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6bc18d-3b39-445f-bc28-917472bc6353 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:37.417606 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 107/223] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:37 2022] GET /identity/v3/users/4ecfa39ed2b44f759641959889c4eb9b => generated 283 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:37.424221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6bc18d-3b39-445f-bc28-917472bc6353 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:37.520111 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:37.550337 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8be3bb-d98a-4603-9ecc-609113996a6c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:37.614426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce06e1f-e359-45f0-adbc-20c9c4e2fdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:37.615122 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce06e1f-e359-45f0-adbc-20c9c4e2fdcd None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.615405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce06e1f-e359-45f0-adbc-20c9c4e2fdcd None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.615684 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce06e1f-e359-45f0-adbc-20c9c4e2fdcd None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.616997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce06e1f-e359-45f0-adbc-20c9c4e2fdcd None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:37.619310 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce06e1f-e359-45f0-adbc-20c9c4e2fdcd None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:37.620795 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 117/224] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:37 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:37.622066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de28bf1-02af-45c4-824e-88d0fce538ad None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.622334 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de28bf1-02af-45c4-824e-88d0fce538ad None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.622573 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de28bf1-02af-45c4-824e-88d0fce538ad None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.624174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 118/225] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:37.629517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de28bf1-02af-45c4-824e-88d0fce538ad None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:37.675078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6bc18d-3b39-445f-bc28-917472bc6353 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:37.730405 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:37.742669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53016af7-d2ed-445a-b82d-73f50dfafe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:37.743788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53016af7-d2ed-445a-b82d-73f50dfafe52 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.744307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53016af7-d2ed-445a-b82d-73f50dfafe52 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.744831 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53016af7-d2ed-445a-b82d-73f50dfafe52 None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.746288 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53016af7-d2ed-445a-b82d-73f50dfafe52 None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:37.748005 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53016af7-d2ed-445a-b82d-73f50dfafe52 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:37.750891 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de28bf1-02af-45c4-824e-88d0fce538ad None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:37.751252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 108/226] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:37 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:37.752914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e4786-6f95-474f-a03f-902463dfd720 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.753428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e4786-6f95-474f-a03f-902463dfd720 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.753905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e4786-6f95-474f-a03f-902463dfd720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.775021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-970e4786-6f95-474f-a03f-902463dfd720 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:37.777119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-970e4786-6f95-474f-a03f-902463dfd720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:37.794764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f532c-437e-40ae-8278-1b2cad83db51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:37.795501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f532c-437e-40ae-8278-1b2cad83db51 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.795785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f532c-437e-40ae-8278-1b2cad83db51 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.796061 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f532c-437e-40ae-8278-1b2cad83db51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.797055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f532c-437e-40ae-8278-1b2cad83db51 None admin] RBAC: Adding query filter params (role.id=c7b826398b4e466491d4723321aaa5f1, scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15, user.id=0b0ae20c19bf407cb727570d3f18b893) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:37.797368 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f532c-437e-40ae-8278-1b2cad83db51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:37.798879 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f532c-437e-40ae-8278-1b2cad83db51 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:37.806008 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 119/227] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:37 2022] GET /identity/v3/role_assignments?user.id=0b0ae20c19bf407cb727570d3f18b893&scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15&role.id=c7b826398b4e466491d4723321aaa5f1 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:37.806999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f532c-437e-40ae-8278-1b2cad83db51 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:37.826490 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:37.869579 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 109/228] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:37.967162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f532c-437e-40ae-8278-1b2cad83db51 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:38.019157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6407d7-824d-4b28-8036-5313b37dcb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:38.019909 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6407d7-824d-4b28-8036-5313b37dcb8e None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:38.020196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6407d7-824d-4b28-8036-5313b37dcb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:38.020504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6407d7-824d-4b28-8036-5313b37dcb8e None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15/users/4ecfa39ed2b44f759641959889c4eb9b/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:38.022612 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6407d7-824d-4b28-8036-5313b37dcb8e None admin] RBAC: Authorizing `identity:create_grant(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15, user_id=4ecfa39ed2b44f759641959889c4eb9b, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:38.024147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6407d7-824d-4b28-8036-5313b37dcb8e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:38.094920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 120/229] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:37 2022] PUT /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15/users/4ecfa39ed2b44f759641959889c4eb9b/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:27:38.347605 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e4786-6f95-474f-a03f-902463dfd720 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:38.466913 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e4786-6f95-474f-a03f-902463dfd720 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:38.496342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:38.511295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65fa33-848a-4231-ba22-d1a2e2a41db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:38.512428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65fa33-848a-4231-ba22-d1a2e2a41db3 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:38.512883 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65fa33-848a-4231-ba22-d1a2e2a41db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:38.514475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65fa33-848a-4231-ba22-d1a2e2a41db3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:38.526708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65fa33-848a-4231-ba22-d1a2e2a41db3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-5c65fa33-848a-4231-ba22-d1a2e2a41db3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:38.532587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:38.533546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:38.534326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:38.534326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:38.534326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:38.534326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:38.534326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:38.534326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:38.534326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:38.534326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:38.534326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:38.534326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:38.534326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:27:38.534326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:38.547391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 110/230] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:38.549929 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6407d7-824d-4b28-8036-5313b37dcb8e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:38.700493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:38.747179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6407d7-824d-4b28-8036-5313b37dcb8e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:38.829810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ef1c5-d5ef-4870-a406-6c351f436265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:38.830882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ef1c5-d5ef-4870-a406-6c351f436265 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:38.831380 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ef1c5-d5ef-4870-a406-6c351f436265 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:38.831856 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ef1c5-d5ef-4870-a406-6c351f436265 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:38.833163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ef1c5-d5ef-4870-a406-6c351f436265 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:38.833693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ef1c5-d5ef-4870-a406-6c351f436265 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:38.836280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ef1c5-d5ef-4870-a406-6c351f436265 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:38.862192 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 121/231] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:38.874993 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65fa33-848a-4231-ba22-d1a2e2a41db3 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:39.100954 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65fa33-848a-4231-ba22-d1a2e2a41db3 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:39.145010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538e3a6-96d8-4a37-8891-5214649e14b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:39.145663 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538e3a6-96d8-4a37-8891-5214649e14b8 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:39.145943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538e3a6-96d8-4a37-8891-5214649e14b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:39.146213 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538e3a6-96d8-4a37-8891-5214649e14b8 None admin] PATH_INFO: `/v3/users/4ecfa39ed2b44f759641959889c4eb9b` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:39.147409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538e3a6-96d8-4a37-8891-5214649e14b8 None admin] RBAC: Authorizing `identity:get_user(user_id=4ecfa39ed2b44f759641959889c4eb9b)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:39.148964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538e3a6-96d8-4a37-8891-5214649e14b8 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:39.151003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 111/232] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:38 2022] GET /identity/v3/users/4ecfa39ed2b44f759641959889c4eb9b => generated 283 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:39.158435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ef1c5-d5ef-4870-a406-6c351f436265 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:39.308895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc8860-b8c0-46ac-aa17-b4e61aacf37b None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:39.309451 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc8860-b8c0-46ac-aa17-b4e61aacf37b None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:39.309867 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc8860-b8c0-46ac-aa17-b4e61aacf37b None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:39.311758 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 112/233] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:39.317609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af762673-8b10-433b-a874-5f0f5575beb0 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:39.318011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af762673-8b10-433b-a874-5f0f5575beb0 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:39.318384 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af762673-8b10-433b-a874-5f0f5575beb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:39.364932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-af762673-8b10-433b-a874-5f0f5575beb0 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:39.366248 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-af762673-8b10-433b-a874-5f0f5575beb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:39.432193 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ef1c5-d5ef-4870-a406-6c351f436265 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:39.481073 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:39.494066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5859a-f009-43e9-904b-563b4edc2a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:39.495191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5859a-f009-43e9-904b-563b4edc2a1f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:39.496046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5859a-f009-43e9-904b-563b4edc2a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:39.496786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5859a-f009-43e9-904b-563b4edc2a1f None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:39.498501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5859a-f009-43e9-904b-563b4edc2a1f None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:39.500750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5859a-f009-43e9-904b-563b4edc2a1f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:39.503054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 122/234] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:39 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:39.510138 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f495f9-c716-4361-8232-c0849cb45a50 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:39.510593 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f495f9-c716-4361-8232-c0849cb45a50 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:39.511002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f495f9-c716-4361-8232-c0849cb45a50 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:39.512533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 123/235] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:39.513533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f495f9-c716-4361-8232-c0849cb45a50 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:39.519340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:39.522720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 113/236] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:39.524669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa954259-6c70-4c9d-a2d3-ecd87515b565 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:39.525459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa954259-6c70-4c9d-a2d3-ecd87515b565 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:39.526414 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa954259-6c70-4c9d-a2d3-ecd87515b565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:39.580288 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-fa954259-6c70-4c9d-a2d3-ecd87515b565 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:39.582285 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-fa954259-6c70-4c9d-a2d3-ecd87515b565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:39.796617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 114/237] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:39.822736 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f495f9-c716-4361-8232-c0849cb45a50 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:39.845892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa954259-6c70-4c9d-a2d3-ecd87515b565 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:39.888185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e4e2b-34db-46c7-9845-349422fa4a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:39.889344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e4e2b-34db-46c7-9845-349422fa4a51 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:39.889973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e4e2b-34db-46c7-9845-349422fa4a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:39.891132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e4e2b-34db-46c7-9845-349422fa4a51 None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15/users/4ecfa39ed2b44f759641959889c4eb9b/roles/003cfd128ecf4c238197991ad36f0c9a` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:39.900197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e4e2b-34db-46c7-9845-349422fa4a51 None admin] RBAC: Authorizing `identity:create_grant(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15, user_id=4ecfa39ed2b44f759641959889c4eb9b, role_id=003cfd128ecf4c238197991ad36f0c9a)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:39.903061 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e4e2b-34db-46c7-9845-349422fa4a51 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:39.980558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa954259-6c70-4c9d-a2d3-ecd87515b565 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:39.997325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:40.020344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 124/238] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:39 2022] PUT /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15/users/4ecfa39ed2b44f759641959889c4eb9b/roles/003cfd128ecf4c238197991ad36f0c9a => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:27:40.031410 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40149aae-5894-4f17-b375-5e0b05b30738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:40.032079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40149aae-5894-4f17-b375-5e0b05b30738 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:40.032362 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40149aae-5894-4f17-b375-5e0b05b30738 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:40.032676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40149aae-5894-4f17-b375-5e0b05b30738 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:40.045789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40149aae-5894-4f17-b375-5e0b05b30738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-40149aae-5894-4f17-b375-5e0b05b30738 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:40.051383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:40.052428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:40.054217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:40.054217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:40.054217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:40.054217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:40.054217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:40.054217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:40.054217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:40.054217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:40.054217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:40.054217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:40.054217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:40.054217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:40.054217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 115/239] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:40.063252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e4e2b-34db-46c7-9845-349422fa4a51 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:40.165793 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:40.221530 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e4e2b-34db-46c7-9845-349422fa4a51 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:40.280251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72e370-189d-40f7-a415-02e618cecf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:40.280943 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72e370-189d-40f7-a415-02e618cecf89 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:40.281249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72e370-189d-40f7-a415-02e618cecf89 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:40.281523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72e370-189d-40f7-a415-02e618cecf89 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:40.282223 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72e370-189d-40f7-a415-02e618cecf89 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:40.282528 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72e370-189d-40f7-a415-02e618cecf89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:40.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72e370-189d-40f7-a415-02e618cecf89 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:40.293687 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 125/240] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:40.300809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40149aae-5894-4f17-b375-5e0b05b30738 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:40.489200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72e370-189d-40f7-a415-02e618cecf89 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:40.551526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40149aae-5894-4f17-b375-5e0b05b30738 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:40.618641 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3121720-95f2-427a-9dbc-84d0a39cca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:40.619303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3121720-95f2-427a-9dbc-84d0a39cca9b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:40.619574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3121720-95f2-427a-9dbc-84d0a39cca9b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:40.619847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3121720-95f2-427a-9dbc-84d0a39cca9b None admin] PATH_INFO: `/v3/users/9da2338217d345b2bc73f4e11aadf382` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:40.641449 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3121720-95f2-427a-9dbc-84d0a39cca9b None admin] RBAC: Authorizing `identity:get_user(user_id=9da2338217d345b2bc73f4e11aadf382)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:40.644477 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3121720-95f2-427a-9dbc-84d0a39cca9b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:40.652511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 116/241] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:40 2022] GET /identity/v3/users/9da2338217d345b2bc73f4e11aadf382 => generated 297 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:40.652511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d29e3c-b5c6-4212-a6ad-7284be4b0078 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:40.652511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d29e3c-b5c6-4212-a6ad-7284be4b0078 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:40.652511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d29e3c-b5c6-4212-a6ad-7284be4b0078 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:40.653383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 117/242] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:40.665148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d29e3c-b5c6-4212-a6ad-7284be4b0078 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:40.716564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72e370-189d-40f7-a415-02e618cecf89 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:40.728892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:40.768920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:40.786676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b02d8c-e02d-4a99-9c8c-cb9e6ca9caef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:40.787731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b02d8c-e02d-4a99-9c8c-cb9e6ca9caef None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:40.788211 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b02d8c-e02d-4a99-9c8c-cb9e6ca9caef None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:40.788702 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b02d8c-e02d-4a99-9c8c-cb9e6ca9caef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:40.790566 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b02d8c-e02d-4a99-9c8c-cb9e6ca9caef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:40.793325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b02d8c-e02d-4a99-9c8c-cb9e6ca9caef None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:40.800495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 126/243] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:40.800495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06beeb-0e7d-4871-894a-6b7e69503073 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:40.800495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06beeb-0e7d-4871-894a-6b7e69503073 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:40.800978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06beeb-0e7d-4871-894a-6b7e69503073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:40.833080 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fa06beeb-0e7d-4871-894a-6b7e69503073 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:40.835074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fa06beeb-0e7d-4871-894a-6b7e69503073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:40.896086 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d29e3c-b5c6-4212-a6ad-7284be4b0078 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:40.966383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff1ce2-953d-48e9-a4ea-3553f077dd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:40.966383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff1ce2-953d-48e9-a4ea-3553f077dd0c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:40.966383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff1ce2-953d-48e9-a4ea-3553f077dd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:40.966383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff1ce2-953d-48e9-a4ea-3553f077dd0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:40.967588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 127/244] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:40.970353 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06beeb-0e7d-4871-894a-6b7e69503073 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:41.147106 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff1ce2-953d-48e9-a4ea-3553f077dd0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:41.150864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff1ce2-953d-48e9-a4ea-3553f077dd0c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:41.209989 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:41.256875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06beeb-0e7d-4871-894a-6b7e69503073 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:41.304825 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:41.324445 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 118/245] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:41.324445 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7c6e0-e057-423f-8e45-4871b1e983aa None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:41.324445 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7c6e0-e057-423f-8e45-4871b1e983aa None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:41.324445 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7c6e0-e057-423f-8e45-4871b1e983aa None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:41.325547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 119/246] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:41.327286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7776899-3562-4d02-bbdb-9a0f20a57a96 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:41.327471 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7776899-3562-4d02-bbdb-9a0f20a57a96 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:41.328440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7776899-3562-4d02-bbdb-9a0f20a57a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:41.339837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369925ae-f5af-448b-bf7a-16a15c50c3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:41.341440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369925ae-f5af-448b-bf7a-16a15c50c3e9 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:41.343250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369925ae-f5af-448b-bf7a-16a15c50c3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:41.344710 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369925ae-f5af-448b-bf7a-16a15c50c3e9 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:41.346657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369925ae-f5af-448b-bf7a-16a15c50c3e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:41.349698 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-f7776899-3562-4d02-bbdb-9a0f20a57a96 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:41.350116 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369925ae-f5af-448b-bf7a-16a15c50c3e9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:41.350935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-f7776899-3562-4d02-bbdb-9a0f20a57a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:41.452547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 128/247] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:27:40 2022] POST /identity/v3/users => generated 293 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:41.452547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369925ae-f5af-448b-bf7a-16a15c50c3e9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:41.474895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 120/248] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:41.481934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7776899-3562-4d02-bbdb-9a0f20a57a96 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:41.604862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:41.630186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369925ae-f5af-448b-bf7a-16a15c50c3e9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:41.642781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7776899-3562-4d02-bbdb-9a0f20a57a96 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:41.686974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b6383-a647-48c8-b986-188293f00c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:41.688318 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b6383-a647-48c8-b986-188293f00c0d None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:41.688843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b6383-a647-48c8-b986-188293f00c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:41.689123 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b6383-a647-48c8-b986-188293f00c0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:41.704908 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b6383-a647-48c8-b986-188293f00c0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-ce8b6383-a647-48c8-b986-188293f00c0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:41.709875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:41.717377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:41.718535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:41.718535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:41.718535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:41.718535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:41.718535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:41.718535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:41.718535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:41.718535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:41.718535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:41.718535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:41.718535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:41.718535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:41.718535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 121/249] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:41.718535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b6383-a647-48c8-b986-188293f00c0d None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:41.739877 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb4643-7b87-4d40-9654-170b1c15fe50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:41.740653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb4643-7b87-4d40-9654-170b1c15fe50 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:41.740935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb4643-7b87-4d40-9654-170b1c15fe50 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:41.741210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb4643-7b87-4d40-9654-170b1c15fe50 None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:41.742429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb4643-7b87-4d40-9654-170b1c15fe50 None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:41.751276 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb4643-7b87-4d40-9654-170b1c15fe50 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:41.754068 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 129/250] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:41 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:41.762020 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb4643-7b87-4d40-9654-170b1c15fe50 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:41.872433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:41.913522 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b6383-a647-48c8-b986-188293f00c0d None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:41.941028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:41.963429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c325e-14ed-4752-bf9d-c8a464f280f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:41.963785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb4643-7b87-4d40-9654-170b1c15fe50 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:41.964630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c325e-14ed-4752-bf9d-c8a464f280f0 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:41.965114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c325e-14ed-4752-bf9d-c8a464f280f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:41.965621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c325e-14ed-4752-bf9d-c8a464f280f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:41.967211 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c325e-14ed-4752-bf9d-c8a464f280f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:41.967794 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c325e-14ed-4752-bf9d-c8a464f280f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:41.971752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c325e-14ed-4752-bf9d-c8a464f280f0 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:41.983807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 122/251] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:41.986038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c325e-14ed-4752-bf9d-c8a464f280f0 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:42.028683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a108ee-fb41-480b-a792-83d1d96df121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:42.029733 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a108ee-fb41-480b-a792-83d1d96df121 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:42.030235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a108ee-fb41-480b-a792-83d1d96df121 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:42.032006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a108ee-fb41-480b-a792-83d1d96df121 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:42.034659 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a108ee-fb41-480b-a792-83d1d96df121 None admin] RBAC: Adding query filter params (role.id=a915b705941d46538bd6a59aac287977, scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15, user.id=9da2338217d345b2bc73f4e11aadf382) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:42.035245 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a108ee-fb41-480b-a792-83d1d96df121 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:42.039282 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a108ee-fb41-480b-a792-83d1d96df121 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:42.050282 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 130/252] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:41 2022] GET /identity/v3/role_assignments?user.id=9da2338217d345b2bc73f4e11aadf382&scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15&role.id=a915b705941d46538bd6a59aac287977 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:42.052623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a108ee-fb41-480b-a792-83d1d96df121 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:42.188137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c325e-14ed-4752-bf9d-c8a464f280f0 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:42.282396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a108ee-fb41-480b-a792-83d1d96df121 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:42.283461 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26796b-e8b2-4b26-a94d-39132f12196d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:42.284425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26796b-e8b2-4b26-a94d-39132f12196d None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:42.284952 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26796b-e8b2-4b26-a94d-39132f12196d None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:42.285440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26796b-e8b2-4b26-a94d-39132f12196d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:42.312898 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26796b-e8b2-4b26-a94d-39132f12196d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-3e26796b-e8b2-4b26-a94d-39132f12196d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:42.319090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:42.320057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:42.320915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:42.320915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:42.320915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:42.320915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:42.320915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:42.320915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:42.320915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:42.320915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:42.320915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:42.320915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:42.320915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:42.320915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:42.322669 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:42.328447 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 123/253] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:42.336178 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26796b-e8b2-4b26-a94d-39132f12196d None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:42.383220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffc1a7-943e-456e-b6f9-74483bc137d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:42.383903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffc1a7-943e-456e-b6f9-74483bc137d2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:42.384187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffc1a7-943e-456e-b6f9-74483bc137d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:42.384483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffc1a7-943e-456e-b6f9-74483bc137d2 None admin] PATH_INFO: `/v3/users/0b0ae20c19bf407cb727570d3f18b893` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:42.385847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffc1a7-943e-456e-b6f9-74483bc137d2 None admin] RBAC: Authorizing `identity:get_user(user_id=0b0ae20c19bf407cb727570d3f18b893)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:42.389145 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffc1a7-943e-456e-b6f9-74483bc137d2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:42.391787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 131/254] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:42 2022] GET /identity/v3/users/0b0ae20c19bf407cb727570d3f18b893 => generated 297 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:42.399724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffc1a7-943e-456e-b6f9-74483bc137d2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:42.431297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:42.560225 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26796b-e8b2-4b26-a94d-39132f12196d None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:42.652222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd502a-81dc-44f3-92c9-e83522e3d750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:42.653910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd502a-81dc-44f3-92c9-e83522e3d750 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:42.654328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd502a-81dc-44f3-92c9-e83522e3d750 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:42.654789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd502a-81dc-44f3-92c9-e83522e3d750 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:42.655741 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd502a-81dc-44f3-92c9-e83522e3d750 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:42.655994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffc1a7-943e-456e-b6f9-74483bc137d2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:42.656538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd502a-81dc-44f3-92c9-e83522e3d750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:42.660466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd502a-81dc-44f3-92c9-e83522e3d750 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:42.680845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 124/255] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:42.693103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd502a-81dc-44f3-92c9-e83522e3d750 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:42.729178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c10e3-a98c-416b-9f8a-c1a8d51e7518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:42.730388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c10e3-a98c-416b-9f8a-c1a8d51e7518 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:42.730931 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c10e3-a98c-416b-9f8a-c1a8d51e7518 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:42.731437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c10e3-a98c-416b-9f8a-c1a8d51e7518 None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:42.733239 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c10e3-a98c-416b-9f8a-c1a8d51e7518 None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:42.742650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c10e3-a98c-416b-9f8a-c1a8d51e7518 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:42.745213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 132/256] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:42 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:42.752025 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c10e3-a98c-416b-9f8a-c1a8d51e7518 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:42.884895 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:42.964198 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd502a-81dc-44f3-92c9-e83522e3d750 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:42.992833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:43.034646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c10e3-a98c-416b-9f8a-c1a8d51e7518 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:43.065628 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb1806-be2b-4a4d-a075-500dc3b8780c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:43.066294 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb1806-be2b-4a4d-a075-500dc3b8780c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:43.066576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb1806-be2b-4a4d-a075-500dc3b8780c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:43.066840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb1806-be2b-4a4d-a075-500dc3b8780c None admin] PATH_INFO: `/v3/users/4ecfa39ed2b44f759641959889c4eb9b` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:43.069015 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb1806-be2b-4a4d-a075-500dc3b8780c None admin] RBAC: Authorizing `identity:get_user(user_id=4ecfa39ed2b44f759641959889c4eb9b)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:43.070485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb1806-be2b-4a4d-a075-500dc3b8780c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:43.074390 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 125/257] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:42 2022] GET /identity/v3/users/4ecfa39ed2b44f759641959889c4eb9b => generated 283 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:43.082864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb1806-be2b-4a4d-a075-500dc3b8780c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:43.121541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37c550-945f-4778-908b-28435a82beed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:43.122233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37c550-945f-4778-908b-28435a82beed None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:43.122513 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37c550-945f-4778-908b-28435a82beed None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:43.122788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37c550-945f-4778-908b-28435a82beed None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15/users/0b0ae20c19bf407cb727570d3f18b893/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:43.126507 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37c550-945f-4778-908b-28435a82beed None admin] RBAC: Authorizing `identity:create_grant(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15, user_id=0b0ae20c19bf407cb727570d3f18b893, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:43.128057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37c550-945f-4778-908b-28435a82beed None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:43.231075 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 133/258] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:42 2022] PUT /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15/users/0b0ae20c19bf407cb727570d3f18b893/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:27:43.232437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef4598-0754-4147-8a02-18529297702e None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:43.232668 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef4598-0754-4147-8a02-18529297702e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:43.233071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef4598-0754-4147-8a02-18529297702e None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:43.236109 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 134/259] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:43.243216 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc8127-3748-47dc-96a0-39d17437e2a1 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:43.243549 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc8127-3748-47dc-96a0-39d17437e2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:43.244028 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc8127-3748-47dc-96a0-39d17437e2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:43.296244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-78cc8127-3748-47dc-96a0-39d17437e2a1 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:43.298563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-78cc8127-3748-47dc-96a0-39d17437e2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:43.311938 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb1806-be2b-4a4d-a075-500dc3b8780c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:43.402357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c165e-3cb0-4640-9971-398f9b2f2265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:43.402987 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c165e-3cb0-4640-9971-398f9b2f2265 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:43.403265 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c165e-3cb0-4640-9971-398f9b2f2265 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:43.408731 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c165e-3cb0-4640-9971-398f9b2f2265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:43.494962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:43.528702 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 135/260] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:43.556295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c165e-3cb0-4640-9971-398f9b2f2265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:43.560016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c165e-3cb0-4640-9971-398f9b2f2265 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:43.737589 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 126/261] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:43.745112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc8127-3748-47dc-96a0-39d17437e2a1 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:43.838252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:43.965654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc8127-3748-47dc-96a0-39d17437e2a1 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:44.050846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0af55-080c-46de-ba37-228c289db23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:44.051802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0af55-080c-46de-ba37-228c289db23a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:44.052274 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0af55-080c-46de-ba37-228c289db23a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:44.052769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0af55-080c-46de-ba37-228c289db23a None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:44.054985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0af55-080c-46de-ba37-228c289db23a None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:44.057865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0af55-080c-46de-ba37-228c289db23a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:44.061655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 136/262] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:43 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:44.062721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c165e-3cb0-4640-9971-398f9b2f2265 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:44.069615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0af55-080c-46de-ba37-228c289db23a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:44.324055 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:44.338882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0af55-080c-46de-ba37-228c289db23a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:44.356831 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:44.464363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c165e-3cb0-4640-9971-398f9b2f2265 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:44.473268 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e6290-f96a-40e6-95ba-c501326913c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:44.473911 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e6290-f96a-40e6-95ba-c501326913c2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:44.474194 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e6290-f96a-40e6-95ba-c501326913c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:44.474469 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e6290-f96a-40e6-95ba-c501326913c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:44.475238 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e6290-f96a-40e6-95ba-c501326913c2 None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15, user.id=4ecfa39ed2b44f759641959889c4eb9b) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:44.475549 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e6290-f96a-40e6-95ba-c501326913c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:44.477081 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e6290-f96a-40e6-95ba-c501326913c2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:44.487006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 137/263] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:44 2022] GET /identity/v3/role_assignments?user.id=4ecfa39ed2b44f759641959889c4eb9b&scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15&role.id=e7f48343fc1542979ab4459865109e30 => generated 608 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:44.591461 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6137b-4950-4746-9649-a11091a6bf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:44.592513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6137b-4950-4746-9649-a11091a6bf59 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:44.593021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6137b-4950-4746-9649-a11091a6bf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:44.594058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6137b-4950-4746-9649-a11091a6bf59 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:44.613958 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6137b-4950-4746-9649-a11091a6bf59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-a0a6137b-4950-4746-9649-a11091a6bf59 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:44.618833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:44.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:44.620583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:44.620583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:44.620583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:44.620583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:44.620583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:44.620583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:44.620583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:44.620583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:44.620583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:44.620583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:44.620583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:27:44.620583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:44.620583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 127/264] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 559 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:44.626380 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e6290-f96a-40e6-95ba-c501326913c2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:44.789737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e6290-f96a-40e6-95ba-c501326913c2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:44.848152 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4aed-29ad-471d-aa43-55af833c97e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:44.849139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4aed-29ad-471d-aa43-55af833c97e9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:44.849613 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4aed-29ad-471d-aa43-55af833c97e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:44.850099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4aed-29ad-471d-aa43-55af833c97e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:44.850973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be4aed-29ad-471d-aa43-55af833c97e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:44.851472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be4aed-29ad-471d-aa43-55af833c97e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:44.853111 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be4aed-29ad-471d-aa43-55af833c97e9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:44.864922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 138/265] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:44.870844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6137b-4950-4746-9649-a11091a6bf59 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:45.142596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6137b-4950-4746-9649-a11091a6bf59 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:45.215061 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:45.224530 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51325874-cc6b-4b33-8d71-f3daa7e6028e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:45.225581 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51325874-cc6b-4b33-8d71-f3daa7e6028e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.226082 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51325874-cc6b-4b33-8d71-f3daa7e6028e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.226573 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51325874-cc6b-4b33-8d71-f3daa7e6028e None admin] PATH_INFO: `/v3/users/4ecfa39ed2b44f759641959889c4eb9b` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.233743 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51325874-cc6b-4b33-8d71-f3daa7e6028e None admin] RBAC: Authorizing `identity:get_user(user_id=4ecfa39ed2b44f759641959889c4eb9b)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:45.236488 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51325874-cc6b-4b33-8d71-f3daa7e6028e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:45.239778 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 128/266] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:44 2022] GET /identity/v3/users/4ecfa39ed2b44f759641959889c4eb9b => generated 283 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:45.250158 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4aed-29ad-471d-aa43-55af833c97e9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:45.311338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:45.413480 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4aed-29ad-471d-aa43-55af833c97e9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:45.449485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efda8ed-78d4-4dd8-9032-55eff00766a4 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.451020 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efda8ed-78d4-4dd8-9032-55eff00766a4 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.451759 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efda8ed-78d4-4dd8-9032-55eff00766a4 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.464438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 129/267] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:45.467310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261d3f2-c5ad-40ff-a679-a73492ba5f13 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.468136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261d3f2-c5ad-40ff-a679-a73492ba5f13 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.469016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261d3f2-c5ad-40ff-a679-a73492ba5f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.473867 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e72f4-0b27-43b4-a414-ecbc1e0efc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:45.474621 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e72f4-0b27-43b4-a414-ecbc1e0efc83 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.474903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e72f4-0b27-43b4-a414-ecbc1e0efc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.475175 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e72f4-0b27-43b4-a414-ecbc1e0efc83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.502259 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-f261d3f2-c5ad-40ff-a679-a73492ba5f13 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:45.504608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-f261d3f2-c5ad-40ff-a679-a73492ba5f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:45.583805 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e72f4-0b27-43b4-a414-ecbc1e0efc83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:45.585579 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e72f4-0b27-43b4-a414-ecbc1e0efc83 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:45.628769 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 130/268] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:45.630732 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382fdba-5df6-4e18-b4a7-4e0a2e79fa13 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.631400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382fdba-5df6-4e18-b4a7-4e0a2e79fa13 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.632273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382fdba-5df6-4e18-b4a7-4e0a2e79fa13 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.638861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 131/269] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:45.643731 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efd15a-41e4-43f5-ac1e-be9a635e0650 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.644694 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efd15a-41e4-43f5-ac1e-be9a635e0650 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.645205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efd15a-41e4-43f5-ac1e-be9a635e0650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.648817 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:45.698543 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-e7efd15a-41e4-43f5-ac1e-be9a635e0650 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:45.700621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e7efd15a-41e4-43f5-ac1e-be9a635e0650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:45.836513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 132/270] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:45.838368 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18dd89-cb26-49df-bb39-22201c70b30d None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.839001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18dd89-cb26-49df-bb39-22201c70b30d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.839422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18dd89-cb26-49df-bb39-22201c70b30d None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.841034 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 133/271] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:45.847381 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d8f74-4e8c-410f-b624-cbfe9fba9a3e None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.847916 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d8f74-4e8c-410f-b624-cbfe9fba9a3e None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.848343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d8f74-4e8c-410f-b624-cbfe9fba9a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.856680 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 139/272] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:45.872181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-932d8f74-4e8c-410f-b624-cbfe9fba9a3e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:45.873788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-932d8f74-4e8c-410f-b624-cbfe9fba9a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:45.882919 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e72f4-0b27-43b4-a414-ecbc1e0efc83 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.019712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:46.053892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 134/273] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:46.144567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d8f74-4e8c-410f-b624-cbfe9fba9a3e None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.153160 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e72f4-0b27-43b4-a414-ecbc1e0efc83 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:46.215097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:46.226614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479b82f-101b-4709-8b92-a8b29051895d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:46.228125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479b82f-101b-4709-8b92-a8b29051895d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.228909 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479b82f-101b-4709-8b92-a8b29051895d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.229617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479b82f-101b-4709-8b92-a8b29051895d None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.232693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479b82f-101b-4709-8b92-a8b29051895d None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:46.236886 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479b82f-101b-4709-8b92-a8b29051895d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:46.241175 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 140/274] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:45 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:46.242964 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479b82f-101b-4709-8b92-a8b29051895d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.352497 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d8f74-4e8c-410f-b624-cbfe9fba9a3e None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:46.400933 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a8e87-9902-4f03-b0c0-7614abdc42fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:46.401720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a8e87-9902-4f03-b0c0-7614abdc42fb None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.401999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a8e87-9902-4f03-b0c0-7614abdc42fb None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.402269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a8e87-9902-4f03-b0c0-7614abdc42fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.415476 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a8e87-9902-4f03-b0c0-7614abdc42fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-df1a8e87-9902-4f03-b0c0-7614abdc42fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:46.420492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:46.422310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:46.423793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:46.423793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:46.423793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:46.423793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:46.423793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:46.423793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:46.423793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:46.423793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:46.423793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:46.423793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:46.423793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:46.423793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:46.423793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 135/275] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:46.423793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a8e87-9902-4f03-b0c0-7614abdc42fb None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.458029 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479b82f-101b-4709-8b92-a8b29051895d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:46.484463 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:46.534884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ed3b2-8029-4799-8185-842d98e30077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:46.535699 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ed3b2-8029-4799-8185-842d98e30077 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.536161 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ed3b2-8029-4799-8185-842d98e30077 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.536558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ed3b2-8029-4799-8185-842d98e30077 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.540500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ed3b2-8029-4799-8185-842d98e30077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:46.541028 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ed3b2-8029-4799-8185-842d98e30077 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:46.544367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 141/276] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:46.545891 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ed3b2-8029-4799-8185-842d98e30077 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.561701 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a8e87-9902-4f03-b0c0-7614abdc42fb None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:46.606377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268b47c-64d5-4679-bae0-a138422660e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:46.607010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268b47c-64d5-4679-bae0-a138422660e6 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.607428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268b47c-64d5-4679-bae0-a138422660e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.607582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268b47c-64d5-4679-bae0-a138422660e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.608340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268b47c-64d5-4679-bae0-a138422660e6 None admin] RBAC: Adding query filter params (role.id=003cfd128ecf4c238197991ad36f0c9a, scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15, user.id=4ecfa39ed2b44f759641959889c4eb9b) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:46.608831 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268b47c-64d5-4679-bae0-a138422660e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:46.610387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268b47c-64d5-4679-bae0-a138422660e6 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:46.624995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 136/277] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:46 2022] GET /identity/v3/role_assignments?user.id=4ecfa39ed2b44f759641959889c4eb9b&scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15&role.id=003cfd128ecf4c238197991ad36f0c9a => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:46.624995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268b47c-64d5-4679-bae0-a138422660e6 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.688031 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:46.734542 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ed3b2-8029-4799-8185-842d98e30077 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:46.775165 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268b47c-64d5-4679-bae0-a138422660e6 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:46.781463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e407bf-b51a-45cf-ac70-e082d9028aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:46.782123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e407bf-b51a-45cf-ac70-e082d9028aeb None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.782470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e407bf-b51a-45cf-ac70-e082d9028aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.782712 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e407bf-b51a-45cf-ac70-e082d9028aeb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.792999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:46.797537 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e407bf-b51a-45cf-ac70-e082d9028aeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-92e407bf-b51a-45cf-ac70-e082d9028aeb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:46.802724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:46.809899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:46.810807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:46.810807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:46.810807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:46.810807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:46.810807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:46.810807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:46.810807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:46.810807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:46.810807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:46.810807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:46.810807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:46.810807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:46.810807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 142/278] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:46.810807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e407bf-b51a-45cf-ac70-e082d9028aeb None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.833598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929074ba-d130-4f0e-9194-9c9095a7cd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:46.834233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929074ba-d130-4f0e-9194-9c9095a7cd7d None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.834529 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929074ba-d130-4f0e-9194-9c9095a7cd7d None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.834788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929074ba-d130-4f0e-9194-9c9095a7cd7d None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.835544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929074ba-d130-4f0e-9194-9c9095a7cd7d None admin] RBAC: Adding query filter params (name=member) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:46.836643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929074ba-d130-4f0e-9194-9c9095a7cd7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:46.837385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929074ba-d130-4f0e-9194-9c9095a7cd7d None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:46.850055 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 137/279] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:46.852272 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0dc6d-be77-4ca7-86e9-5428f92d97b5 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.852915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0dc6d-be77-4ca7-86e9-5428f92d97b5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.853409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0dc6d-be77-4ca7-86e9-5428f92d97b5 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.855978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 138/280] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:46.862176 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0dc6d-be77-4ca7-86e9-5428f92d97b5 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.962180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e407bf-b51a-45cf-ac70-e082d9028aeb None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:47.013263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:47.013505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0dc6d-be77-4ca7-86e9-5428f92d97b5 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:47.027151 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a15986-53c1-4b6b-ad6d-6cd163cb4be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:47.028179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a15986-53c1-4b6b-ad6d-6cd163cb4be3 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:47.028700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a15986-53c1-4b6b-ad6d-6cd163cb4be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:47.029169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a15986-53c1-4b6b-ad6d-6cd163cb4be3 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:47.030372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a15986-53c1-4b6b-ad6d-6cd163cb4be3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:47.033342 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a15986-53c1-4b6b-ad6d-6cd163cb4be3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:47.052205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da818d1-f00a-45e2-9325-2fa1c3f318b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:47.052876 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da818d1-f00a-45e2-9325-2fa1c3f318b9 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:47.053168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da818d1-f00a-45e2-9325-2fa1c3f318b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:47.053445 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da818d1-f00a-45e2-9325-2fa1c3f318b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:47.054127 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da818d1-f00a-45e2-9325-2fa1c3f318b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:47.054524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da818d1-f00a-45e2-9325-2fa1c3f318b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:47.055945 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da818d1-f00a-45e2-9325-2fa1c3f318b9 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:47.066704 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 139/281] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:47.067599 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da818d1-f00a-45e2-9325-2fa1c3f318b9 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:47.113172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:47.131650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 143/282] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:27:46 2022] POST /identity/v3/users => generated 293 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:47.133823 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa84e8-31a4-4604-9398-11b079ff2e62 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:47.134284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa84e8-31a4-4604-9398-11b079ff2e62 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:47.134703 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa84e8-31a4-4604-9398-11b079ff2e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:47.171764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-08aa84e8-31a4-4604-9398-11b079ff2e62 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:47.171764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-08aa84e8-31a4-4604-9398-11b079ff2e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:47.209347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da818d1-f00a-45e2-9325-2fa1c3f318b9 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:47.274332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f79d5-7d90-467d-849b-961f61236547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:47.275188 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f79d5-7d90-467d-849b-961f61236547 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:47.275370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f79d5-7d90-467d-849b-961f61236547 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:47.275619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f79d5-7d90-467d-849b-961f61236547 None admin] PATH_INFO: `/v3/users/03d13266d68141a3b919dae115eb51e9` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:47.285177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 144/283] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:47.286826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa84e8-31a4-4604-9398-11b079ff2e62 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:47.306733 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f79d5-7d90-467d-849b-961f61236547 None admin] RBAC: Authorizing `identity:get_user(user_id=03d13266d68141a3b919dae115eb51e9)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:47.308226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f79d5-7d90-467d-849b-961f61236547 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:47.310921 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 140/284] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:47 2022] GET /identity/v3/users/03d13266d68141a3b919dae115eb51e9 => generated 293 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:47.317972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f79d5-7d90-467d-849b-961f61236547 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:47.484305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:47.490154 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa84e8-31a4-4604-9398-11b079ff2e62 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:47.501624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f79d5-7d90-467d-849b-961f61236547 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:47.508871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:47.555270 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb8018-cd22-4311-8971-8511cd76d650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:47.556383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb8018-cd22-4311-8971-8511cd76d650 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:47.556962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb8018-cd22-4311-8971-8511cd76d650 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:47.557465 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb8018-cd22-4311-8971-8511cd76d650 None admin] PATH_INFO: `/v3/users/9da2338217d345b2bc73f4e11aadf382` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:47.560200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb8018-cd22-4311-8971-8511cd76d650 None admin] RBAC: Authorizing `identity:get_user(user_id=9da2338217d345b2bc73f4e11aadf382)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:47.563374 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb8018-cd22-4311-8971-8511cd76d650 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:47.572855 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 145/285] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:47 2022] GET /identity/v3/users/9da2338217d345b2bc73f4e11aadf382 => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:47.578837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb8018-cd22-4311-8971-8511cd76d650 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:47.584209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e37ce3-a6ae-4cbe-90d0-5e215f3d2337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:47.584989 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e37ce3-a6ae-4cbe-90d0-5e215f3d2337 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:47.585267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e37ce3-a6ae-4cbe-90d0-5e215f3d2337 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:47.585583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e37ce3-a6ae-4cbe-90d0-5e215f3d2337 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:47.587337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e37ce3-a6ae-4cbe-90d0-5e215f3d2337 None admin] RBAC: Adding query filter params (role.id=c7b826398b4e466491d4723321aaa5f1, user.id=03d13266d68141a3b919dae115eb51e9, scope.system=all) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:47.587665 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e37ce3-a6ae-4cbe-90d0-5e215f3d2337 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:47.589193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e37ce3-a6ae-4cbe-90d0-5e215f3d2337 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:47.600562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 141/286] 10.222.0.70 () {56 vars in 1308 bytes} [Wed Aug 24 16:27:47 2022] GET /identity/v3/role_assignments?user.id=03d13266d68141a3b919dae115eb51e9&scope.system=all&role.id=c7b826398b4e466491d4723321aaa5f1 => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:47.693643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e37ce3-a6ae-4cbe-90d0-5e215f3d2337 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:47.760267 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb8018-cd22-4311-8971-8511cd76d650 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:47.809116 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a02b06-343f-412a-abd1-fa936203ac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:47.809767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a02b06-343f-412a-abd1-fa936203ac48 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:47.810053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a02b06-343f-412a-abd1-fa936203ac48 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:47.810321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a02b06-343f-412a-abd1-fa936203ac48 None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:47.814538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a02b06-343f-412a-abd1-fa936203ac48 None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:47.815616 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a02b06-343f-412a-abd1-fa936203ac48 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:47.817518 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 146/287] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:47 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:47.826233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a02b06-343f-412a-abd1-fa936203ac48 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:47.904444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:47.912219 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e37ce3-a6ae-4cbe-90d0-5e215f3d2337 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:47.986517 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0654c-d7d9-413a-8428-7f47fa181485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:47.987666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0654c-d7d9-413a-8428-7f47fa181485 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:47.988159 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0654c-d7d9-413a-8428-7f47fa181485 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:47.989099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0654c-d7d9-413a-8428-7f47fa181485 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:48.028593 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d0654c-d7d9-413a-8428-7f47fa181485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-f1d0654c-d7d9-413a-8428-7f47fa181485 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:48.048359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:48.049465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:48.050373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:48.050373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:48.050373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:48.050373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:48.050373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:48.050373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:48.050373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:48.050373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:48.050373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:48.050373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:48.050373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:48.050373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:48.051229 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 142/288] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:48.057672 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a02b06-343f-412a-abd1-fa936203ac48 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:48.062739 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0654c-d7d9-413a-8428-7f47fa181485 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:48.116670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:48.147642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98436c-51d7-499b-a56c-42e373617226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:48.149085 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98436c-51d7-499b-a56c-42e373617226 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:48.149804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98436c-51d7-499b-a56c-42e373617226 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:48.150391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98436c-51d7-499b-a56c-42e373617226 None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15/users/9da2338217d345b2bc73f4e11aadf382/roles/a915b705941d46538bd6a59aac287977` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:48.154180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98436c-51d7-499b-a56c-42e373617226 None admin] RBAC: Authorizing `identity:create_grant(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15, user_id=9da2338217d345b2bc73f4e11aadf382, role_id=a915b705941d46538bd6a59aac287977)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:48.157861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98436c-51d7-499b-a56c-42e373617226 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:48.247305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0654c-d7d9-413a-8428-7f47fa181485 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:48.249487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 147/289] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:47 2022] PUT /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15/users/9da2338217d345b2bc73f4e11aadf382/roles/a915b705941d46538bd6a59aac287977 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:27:48.296971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706db01-58cf-40f9-80a8-5739a8fb28c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:48.297550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706db01-58cf-40f9-80a8-5739a8fb28c7 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:48.297848 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706db01-58cf-40f9-80a8-5739a8fb28c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:48.298111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706db01-58cf-40f9-80a8-5739a8fb28c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:48.298829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706db01-58cf-40f9-80a8-5739a8fb28c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:48.299138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706db01-58cf-40f9-80a8-5739a8fb28c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:48.300651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706db01-58cf-40f9-80a8-5739a8fb28c7 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:48.311356 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 143/290] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:48.318513 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98436c-51d7-499b-a56c-42e373617226 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:48.541879 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98436c-51d7-499b-a56c-42e373617226 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:48.563053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:48.608219 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60406c94-e95e-42c2-86f6-1532da99bd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:48.608974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60406c94-e95e-42c2-86f6-1532da99bd0d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:48.609263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60406c94-e95e-42c2-86f6-1532da99bd0d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:48.609540 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60406c94-e95e-42c2-86f6-1532da99bd0d None admin] PATH_INFO: `/v3/users/0b0ae20c19bf407cb727570d3f18b893` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:48.611778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60406c94-e95e-42c2-86f6-1532da99bd0d None admin] RBAC: Authorizing `identity:get_user(user_id=0b0ae20c19bf407cb727570d3f18b893)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:48.613448 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60406c94-e95e-42c2-86f6-1532da99bd0d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:48.615280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 148/291] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:48 2022] GET /identity/v3/users/0b0ae20c19bf407cb727570d3f18b893 => generated 297 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:48.621665 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706db01-58cf-40f9-80a8-5739a8fb28c7 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:48.725571 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:48.746605 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706db01-58cf-40f9-80a8-5739a8fb28c7 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:48.796503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec2683-b2c3-4fee-81f0-b9b394ca2b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:48.797156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec2683-b2c3-4fee-81f0-b9b394ca2b99 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:48.797439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec2683-b2c3-4fee-81f0-b9b394ca2b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:48.797720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec2683-b2c3-4fee-81f0-b9b394ca2b99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:48.902815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec2683-b2c3-4fee-81f0-b9b394ca2b99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:48.904324 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec2683-b2c3-4fee-81f0-b9b394ca2b99 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:49.020893 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:49.050573 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 144/292] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:49.057007 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60406c94-e95e-42c2-86f6-1532da99bd0d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:49.202785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60406c94-e95e-42c2-86f6-1532da99bd0d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:49.243949 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bfedc-bcbe-45af-9801-bcda4e6a94ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:49.245045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bfedc-bcbe-45af-9801-bcda4e6a94ef None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:49.245526 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bfedc-bcbe-45af-9801-bcda4e6a94ef None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:49.245993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bfedc-bcbe-45af-9801-bcda4e6a94ef None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:49.247480 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bfedc-bcbe-45af-9801-bcda4e6a94ef None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:49.249225 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bfedc-bcbe-45af-9801-bcda4e6a94ef None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:49.251218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 149/293] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:49 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:49.257418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec2683-b2c3-4fee-81f0-b9b394ca2b99 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:49.414044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec2683-b2c3-4fee-81f0-b9b394ca2b99 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:49.466331 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d9bbb-5389-4ea5-88ce-3aef8374327c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:49.466864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d9bbb-5389-4ea5-88ce-3aef8374327c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:49.467152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d9bbb-5389-4ea5-88ce-3aef8374327c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:49.467429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d9bbb-5389-4ea5-88ce-3aef8374327c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:49.468766 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d9bbb-5389-4ea5-88ce-3aef8374327c None admin] RBAC: Adding query filter params (role.id=c7b826398b4e466491d4723321aaa5f1, scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15, user.id=0b0ae20c19bf407cb727570d3f18b893) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:49.469087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d9bbb-5389-4ea5-88ce-3aef8374327c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:49.470619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d9bbb-5389-4ea5-88ce-3aef8374327c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:49.482722 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 145/294] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:49 2022] GET /identity/v3/role_assignments?user.id=0b0ae20c19bf407cb727570d3f18b893&scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15&role.id=c7b826398b4e466491d4723321aaa5f1 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:49.998321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8154bf6-c1e0-47c6-aa6f-0184198d2def None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.000518 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8154bf6-c1e0-47c6-aa6f-0184198d2def None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.001319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8154bf6-c1e0-47c6-aa6f-0184198d2def None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.003819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 150/295] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:50.012045 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e54871-fd92-49eb-aeb2-874cc4fb8d61 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.012908 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e54871-fd92-49eb-aeb2-874cc4fb8d61 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.012908 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e54871-fd92-49eb-aeb2-874cc4fb8d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.037022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-79e54871-fd92-49eb-aeb2-874cc4fb8d61 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:50.038343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-79e54871-fd92-49eb-aeb2-874cc4fb8d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:50.096801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:50.135693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 146/296] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:50.211383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00925d-e8eb-4eb7-bf9c-84d9a5eb767b None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.211995 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00925d-e8eb-4eb7-bf9c-84d9a5eb767b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.212816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00925d-e8eb-4eb7-bf9c-84d9a5eb767b None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.215397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 151/297] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:50.221367 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c6ecb-08c9-47da-8b38-0980d2bf69a6 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.222024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c6ecb-08c9-47da-8b38-0980d2bf69a6 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.222024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c6ecb-08c9-47da-8b38-0980d2bf69a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.245469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-8d7c6ecb-08c9-47da-8b38-0980d2bf69a6 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:50.245469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-8d7c6ecb-08c9-47da-8b38-0980d2bf69a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:50.324506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 147/298] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:50.349398 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134471e-55e7-4127-b021-fd9e5904c0c2 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.349967 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134471e-55e7-4127-b021-fd9e5904c0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.350426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134471e-55e7-4127-b021-fd9e5904c0c2 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.352029 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 152/299] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:50.361192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b08d07-a0d6-452a-9c09-44cb043f0702 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.361834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b08d07-a0d6-452a-9c09-44cb043f0702 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.362320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b08d07-a0d6-452a-9c09-44cb043f0702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.382196 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-83b08d07-a0d6-452a-9c09-44cb043f0702 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:50.383793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-83b08d07-a0d6-452a-9c09-44cb043f0702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:50.428994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134471e-55e7-4127-b021-fd9e5904c0c2 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:50.464875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 148/300] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:50.500731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:50.599514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134471e-55e7-4127-b021-fd9e5904c0c2 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:50.668131 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c973973-4915-4721-b2b8-aa9209a8cd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:50.669434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c973973-4915-4721-b2b8-aa9209a8cd87 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.669936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c973973-4915-4721-b2b8-aa9209a8cd87 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.670356 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b08d07-a0d6-452a-9c09-44cb043f0702 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:50.670480 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c973973-4915-4721-b2b8-aa9209a8cd87 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.691659 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c973973-4915-4721-b2b8-aa9209a8cd87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-7c973973-4915-4721-b2b8-aa9209a8cd87 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:50.700944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:50.701932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:50.702766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:50.702766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:50.702766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:50.702766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:50.702766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:50.702766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:50.702766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:50.702766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:50.702766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:50.702766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:50.702766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:50.702766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:50.703375 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 153/301] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:50.712859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c973973-4915-4721-b2b8-aa9209a8cd87 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:50.719108 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:50.802007 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b08d07-a0d6-452a-9c09-44cb043f0702 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:50.842779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617b001-a14d-4892-8cf7-74e36b85fe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:50.844049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617b001-a14d-4892-8cf7-74e36b85fe4e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.844329 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617b001-a14d-4892-8cf7-74e36b85fe4e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.844698 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617b001-a14d-4892-8cf7-74e36b85fe4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.856201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617b001-a14d-4892-8cf7-74e36b85fe4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-4617b001-a14d-4892-8cf7-74e36b85fe4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:50.861158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:50.863224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:50.865348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:50.865348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:50.865348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:50.865348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:50.865348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:50.865348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:50.865348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:50.865348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:50.865348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:50.865348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:50.865348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:50.865348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:50.865348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 149/302] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:50.865348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c117a1-450e-496f-a392-fecef78f756d None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.869640 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c973973-4915-4721-b2b8-aa9209a8cd87 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:50.869772 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c117a1-450e-496f-a392-fecef78f756d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.869772 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c117a1-450e-496f-a392-fecef78f756d None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.869772 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 150/303] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:50.869772 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c117a1-450e-496f-a392-fecef78f756d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:50.881202 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:50.922950 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e854d95-2118-435e-9ab3-40a14764a940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:50.923573 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e854d95-2118-435e-9ab3-40a14764a940 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.923853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e854d95-2118-435e-9ab3-40a14764a940 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.924123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e854d95-2118-435e-9ab3-40a14764a940 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.924848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e854d95-2118-435e-9ab3-40a14764a940 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:50.925148 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e854d95-2118-435e-9ab3-40a14764a940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:50.926581 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e854d95-2118-435e-9ab3-40a14764a940 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:50.935734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 154/304] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:50.937407 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e854d95-2118-435e-9ab3-40a14764a940 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:50.994924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:51.002904 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c117a1-450e-496f-a392-fecef78f756d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:51.041738 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7ed1a-bbba-4092-861e-ece12c9ffe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:51.042815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7ed1a-bbba-4092-861e-ece12c9ffe9a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:51.043094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7ed1a-bbba-4092-861e-ece12c9ffe9a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:51.043367 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7ed1a-bbba-4092-861e-ece12c9ffe9a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:51.044614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7ed1a-bbba-4092-861e-ece12c9ffe9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:51.046126 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7ed1a-bbba-4092-861e-ece12c9ffe9a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:51.047800 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 151/305] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:51.049114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8c541-56a5-4059-b5c2-2252661c668e None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:51.049352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8c541-56a5-4059-b5c2-2252661c668e None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:51.049629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8c541-56a5-4059-b5c2-2252661c668e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:51.055326 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e854d95-2118-435e-9ab3-40a14764a940 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:51.074459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-11d8c541-56a5-4059-b5c2-2252661c668e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:51.076364 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-11d8c541-56a5-4059-b5c2-2252661c668e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:51.105010 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033497f3-e681-47d3-98c7-3a4bf59f8765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:51.105860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033497f3-e681-47d3-98c7-3a4bf59f8765 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:51.106350 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033497f3-e681-47d3-98c7-3a4bf59f8765 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:51.106750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033497f3-e681-47d3-98c7-3a4bf59f8765 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:51.107586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033497f3-e681-47d3-98c7-3a4bf59f8765 None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:51.107898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033497f3-e681-47d3-98c7-3a4bf59f8765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:51.109723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033497f3-e681-47d3-98c7-3a4bf59f8765 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:51.120884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 155/306] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:51.122055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033497f3-e681-47d3-98c7-3a4bf59f8765 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:51.160191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:51.166276 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 152/307] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:51.170009 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8c541-56a5-4059-b5c2-2252661c668e None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:51.258910 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033497f3-e681-47d3-98c7-3a4bf59f8765 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:51.288824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:51.296560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8c541-56a5-4059-b5c2-2252661c668e None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:51.321631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc644f3-0497-4cd1-bcd0-757084bf8131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:51.323093 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc644f3-0497-4cd1-bcd0-757084bf8131 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:51.323590 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc644f3-0497-4cd1-bcd0-757084bf8131 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:51.324063 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc644f3-0497-4cd1-bcd0-757084bf8131 None admin] PATH_INFO: `/v3/users/62d476ab9f20435b8ffb33994ef4b92f` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:51.337257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3554ba-8fd0-470d-9b84-5ec52190e4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:51.337887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3554ba-8fd0-470d-9b84-5ec52190e4f8 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:51.338168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3554ba-8fd0-470d-9b84-5ec52190e4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:51.338443 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3554ba-8fd0-470d-9b84-5ec52190e4f8 None admin] PATH_INFO: `/v3/groups` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:51.339081 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3554ba-8fd0-470d-9b84-5ec52190e4f8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:51.340719 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3554ba-8fd0-470d-9b84-5ec52190e4f8 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:51.350760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc644f3-0497-4cd1-bcd0-757084bf8131 None admin] RBAC: Authorizing `identity:get_user(user_id=62d476ab9f20435b8ffb33994ef4b92f)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:51.353589 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc644f3-0497-4cd1-bcd0-757084bf8131 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:51.361598 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 156/308] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:51 2022] GET /identity/v3/users/62d476ab9f20435b8ffb33994ef4b92f => generated 293 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:51.361598 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc644f3-0497-4cd1-bcd0-757084bf8131 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:51.413288 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 153/309] 10.222.0.70 () {60 vars in 1134 bytes} [Wed Aug 24 16:27:51 2022] POST /identity/v3/groups => generated 229 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:51.414571 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3554ba-8fd0-470d-9b84-5ec52190e4f8 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:51.500164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:51.515900 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc644f3-0497-4cd1-bcd0-757084bf8131 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:51.556072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3554ba-8fd0-470d-9b84-5ec52190e4f8 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:51.564944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5618dee-e6d4-4c1a-9075-fc55f054a1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:51.565914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5618dee-e6d4-4c1a-9075-fc55f054a1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:51.566350 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5618dee-e6d4-4c1a-9075-fc55f054a1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:51.566796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5618dee-e6d4-4c1a-9075-fc55f054a1b4 None admin] PATH_INFO: `/v3/users/03d13266d68141a3b919dae115eb51e9` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:51.568332 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5618dee-e6d4-4c1a-9075-fc55f054a1b4 None admin] RBAC: Authorizing `identity:get_user(user_id=03d13266d68141a3b919dae115eb51e9)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:51.570902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5618dee-e6d4-4c1a-9075-fc55f054a1b4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:51.573606 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 157/310] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:51 2022] GET /identity/v3/users/03d13266d68141a3b919dae115eb51e9 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:51.575476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5618dee-e6d4-4c1a-9075-fc55f054a1b4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:51.608420 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebbde2-c4ae-467c-a1a3-d350414a38ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:51.610413 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebbde2-c4ae-467c-a1a3-d350414a38ae None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:51.610413 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebbde2-c4ae-467c-a1a3-d350414a38ae None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:51.610767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebbde2-c4ae-467c-a1a3-d350414a38ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:51.612293 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebbde2-c4ae-467c-a1a3-d350414a38ae None admin] RBAC: Adding query filter params (role.id=a915b705941d46538bd6a59aac287977, user.id=62d476ab9f20435b8ffb33994ef4b92f, scope.system=all) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:51.612824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebbde2-c4ae-467c-a1a3-d350414a38ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:51.615680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebbde2-c4ae-467c-a1a3-d350414a38ae None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:51.625649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 154/311] 10.222.0.70 () {56 vars in 1308 bytes} [Wed Aug 24 16:27:51 2022] GET /identity/v3/role_assignments?user.id=62d476ab9f20435b8ffb33994ef4b92f&scope.system=all&role.id=a915b705941d46538bd6a59aac287977 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:51.626848 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebbde2-c4ae-467c-a1a3-d350414a38ae None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:51.676836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:51.739057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5618dee-e6d4-4c1a-9075-fc55f054a1b4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:51.781487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7bbe6-79f6-4b61-8cb5-18a5b856887b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:51.782059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7bbe6-79f6-4b61-8cb5-18a5b856887b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:51.782408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7bbe6-79f6-4b61-8cb5-18a5b856887b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:51.782751 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7bbe6-79f6-4b61-8cb5-18a5b856887b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:51.788931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebbde2-c4ae-467c-a1a3-d350414a38ae None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:51.797306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7bbe6-79f6-4b61-8cb5-18a5b856887b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-87d7bbe6-79f6-4b61-8cb5-18a5b856887b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:51.802719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:51.803681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:51.804536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:51.804536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:51.804536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:51.804536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:51.804536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:51.804536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:51.804536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:51.804536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:51.804536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:51.804536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:51.804536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:51.804536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:51.804536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 158/312] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:51.810502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7bbe6-79f6-4b61-8cb5-18a5b856887b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:51.833241 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:51.834379 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84376e11-1be9-4941-977e-c649866dd272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:51.835358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84376e11-1be9-4941-977e-c649866dd272 None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:51.835866 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84376e11-1be9-4941-977e-c649866dd272 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:51.836368 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84376e11-1be9-4941-977e-c649866dd272 None admin] PATH_INFO: `/v3/system/users/03d13266d68141a3b919dae115eb51e9/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:51.838299 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84376e11-1be9-4941-977e-c649866dd272 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=03d13266d68141a3b919dae115eb51e9, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:51.840124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84376e11-1be9-4941-977e-c649866dd272 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:51.928964 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7bbe6-79f6-4b61-8cb5-18a5b856887b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:51.938684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 155/313] 10.222.0.70 () {58 vars in 1334 bytes} [Wed Aug 24 16:27:51 2022] PUT /identity/v3/system/users/03d13266d68141a3b919dae115eb51e9/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:27:51.976145 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539773f-25b3-48f0-b113-61a7201b9ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:51.976863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539773f-25b3-48f0-b113-61a7201b9ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:51.977143 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539773f-25b3-48f0-b113-61a7201b9ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:51.977480 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539773f-25b3-48f0-b113-61a7201b9ab0 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:51.978178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539773f-25b3-48f0-b113-61a7201b9ab0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:51.978571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539773f-25b3-48f0-b113-61a7201b9ab0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:51.980123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539773f-25b3-48f0-b113-61a7201b9ab0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:51.991496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 159/314] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:51.997869 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84376e11-1be9-4941-977e-c649866dd272 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:52.106163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:52.129522 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84376e11-1be9-4941-977e-c649866dd272 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:52.174964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39585fb3-16d1-4ace-9125-f91ef42e11ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:52.175618 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39585fb3-16d1-4ace-9125-f91ef42e11ab None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:52.175901 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39585fb3-16d1-4ace-9125-f91ef42e11ab None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:52.176172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39585fb3-16d1-4ace-9125-f91ef42e11ab None admin] PATH_INFO: `/v3/users/9da2338217d345b2bc73f4e11aadf382` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:52.177594 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39585fb3-16d1-4ace-9125-f91ef42e11ab None admin] RBAC: Authorizing `identity:get_user(user_id=9da2338217d345b2bc73f4e11aadf382)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:52.179271 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39585fb3-16d1-4ace-9125-f91ef42e11ab None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:52.181310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 156/315] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:51 2022] GET /identity/v3/users/9da2338217d345b2bc73f4e11aadf382 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:52.187095 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539773f-25b3-48f0-b113-61a7201b9ab0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:52.279705 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:52.304753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539773f-25b3-48f0-b113-61a7201b9ab0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:52.345371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42641e9-a6fe-4f17-8648-3f2e12eb2d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:52.346299 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42641e9-a6fe-4f17-8648-3f2e12eb2d53 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:52.346788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42641e9-a6fe-4f17-8648-3f2e12eb2d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:52.347386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42641e9-a6fe-4f17-8648-3f2e12eb2d53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:52.424345 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42641e9-a6fe-4f17-8648-3f2e12eb2d53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:52.426155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42641e9-a6fe-4f17-8648-3f2e12eb2d53 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:52.521055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:52.563371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 160/316] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:52.569148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39585fb3-16d1-4ace-9125-f91ef42e11ab None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:52.682882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39585fb3-16d1-4ace-9125-f91ef42e11ab None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:52.722645 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26ef94-0881-4a1b-a7ec-e084d6d9d8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:52.723300 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26ef94-0881-4a1b-a7ec-e084d6d9d8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:52.723581 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26ef94-0881-4a1b-a7ec-e084d6d9d8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:52.723890 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26ef94-0881-4a1b-a7ec-e084d6d9d8e9 None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:52.725115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26ef94-0881-4a1b-a7ec-e084d6d9d8e9 None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:52.726575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26ef94-0881-4a1b-a7ec-e084d6d9d8e9 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:52.733850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 157/317] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:52 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:52.736230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42641e9-a6fe-4f17-8648-3f2e12eb2d53 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:52.851296 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42641e9-a6fe-4f17-8648-3f2e12eb2d53 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:52.890989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8319b-511d-4a18-b10e-4b53ec8f73bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:52.892221 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8319b-511d-4a18-b10e-4b53ec8f73bb None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:52.892763 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8319b-511d-4a18-b10e-4b53ec8f73bb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:52.893241 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8319b-511d-4a18-b10e-4b53ec8f73bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:52.894221 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8319b-511d-4a18-b10e-4b53ec8f73bb None admin] RBAC: Adding query filter params (role.id=a915b705941d46538bd6a59aac287977, scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15, user.id=9da2338217d345b2bc73f4e11aadf382) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:52.894737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8319b-511d-4a18-b10e-4b53ec8f73bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:52.896466 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8319b-511d-4a18-b10e-4b53ec8f73bb None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:52.904116 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 161/318] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:52 2022] GET /identity/v3/role_assignments?user.id=9da2338217d345b2bc73f4e11aadf382&scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15&role.id=a915b705941d46538bd6a59aac287977 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:53.657636 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97b24b-6d46-4371-86db-7ac6572483ef None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:53.658152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97b24b-6d46-4371-86db-7ac6572483ef None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:53.658654 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97b24b-6d46-4371-86db-7ac6572483ef None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:53.660149 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 158/319] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:53.665938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9c389-3274-4232-ad1b-d39f6097c821 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:53.666490 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9c389-3274-4232-ad1b-d39f6097c821 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:53.666490 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9c389-3274-4232-ad1b-d39f6097c821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:53.759290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-81e9c389-3274-4232-ad1b-d39f6097c821 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:53.760556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-81e9c389-3274-4232-ad1b-d39f6097c821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:53.809911 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:53.872087 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 162/320] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:53.917652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695b0a3-70ec-4957-a5e9-6f1eb4430a48 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:53.918055 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695b0a3-70ec-4957-a5e9-6f1eb4430a48 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:53.918478 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695b0a3-70ec-4957-a5e9-6f1eb4430a48 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:53.920912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 159/321] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:53.929132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81277b-b8ad-4312-ad1b-e4af18e96c98 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:53.929403 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81277b-b8ad-4312-ad1b-e4af18e96c98 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:53.929703 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81277b-b8ad-4312-ad1b-e4af18e96c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:53.950164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5c81277b-b8ad-4312-ad1b-e4af18e96c98 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:53.951458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5c81277b-b8ad-4312-ad1b-e4af18e96c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:54.029962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fbe0c-c8c4-43bc-a2a9-5f297722a73c None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:54.030339 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fbe0c-c8c4-43bc-a2a9-5f297722a73c None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:54.031061 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fbe0c-c8c4-43bc-a2a9-5f297722a73c None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:54.032662 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 160/322] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:54.037879 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 163/323] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:54.038138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e741e9-7439-4271-bf51-f3a18a069029 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:54.038384 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e741e9-7439-4271-bf51-f3a18a069029 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:54.038885 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e741e9-7439-4271-bf51-f3a18a069029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:54.058830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-97e741e9-7439-4271-bf51-f3a18a069029 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:54.060122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-97e741e9-7439-4271-bf51-f3a18a069029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:54.077119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:54.151548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 161/324] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:54.237600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81277b-b8ad-4312-ad1b-e4af18e96c98 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:54.312240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e741e9-7439-4271-bf51-f3a18a069029 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:54.358555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81277b-b8ad-4312-ad1b-e4af18e96c98 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:54.361234 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:54.399598 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac7496-6aa1-4fcb-86cd-ca26d1778220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:54.400388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac7496-6aa1-4fcb-86cd-ca26d1778220 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:54.400704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac7496-6aa1-4fcb-86cd-ca26d1778220 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:54.400981 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac7496-6aa1-4fcb-86cd-ca26d1778220 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:54.413372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac7496-6aa1-4fcb-86cd-ca26d1778220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-55ac7496-6aa1-4fcb-86cd-ca26d1778220 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:54.418178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:54.419187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:54.420255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:54.420255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:54.420255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:54.420255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:54.420255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:54.420255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:54.420255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:54.420255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:54.420255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:54.420255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:54.420255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:27:54.420255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:54.420255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 164/325] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:54.422356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac7496-6aa1-4fcb-86cd-ca26d1778220 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:54.457599 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e741e9-7439-4271-bf51-f3a18a069029 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:54.494374 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246ad98-cd40-4e33-8dff-19f48fa48120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:54.495008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246ad98-cd40-4e33-8dff-19f48fa48120 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:54.495287 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246ad98-cd40-4e33-8dff-19f48fa48120 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:54.495560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246ad98-cd40-4e33-8dff-19f48fa48120 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:54.496717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246ad98-cd40-4e33-8dff-19f48fa48120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:54.498168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246ad98-cd40-4e33-8dff-19f48fa48120 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:54.499770 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 162/326] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:54.500616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246ad98-cd40-4e33-8dff-19f48fa48120 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:54.531899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:54.548149 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac7496-6aa1-4fcb-86cd-ca26d1778220 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:54.590232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee559386-061d-48d5-a27b-c65ef051b5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:54.591223 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee559386-061d-48d5-a27b-c65ef051b5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:54.591695 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee559386-061d-48d5-a27b-c65ef051b5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:54.592154 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee559386-061d-48d5-a27b-c65ef051b5b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:54.606846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee559386-061d-48d5-a27b-c65ef051b5b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-ee559386-061d-48d5-a27b-c65ef051b5b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:54.612232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:54.613399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:54.614411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:54.614411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:54.614411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:54.614411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:54.614411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:54.614411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:54.614411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:54.614411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:54.614411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:54.614411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:54.614411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:54.614411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:54.614411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 165/327] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:54.614411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee559386-061d-48d5-a27b-c65ef051b5b9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:54.630140 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246ad98-cd40-4e33-8dff-19f48fa48120 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:54.657878 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:54.679211 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d2f58-a366-4523-8839-e65311db6bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:54.679847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d2f58-a366-4523-8839-e65311db6bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:54.680139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d2f58-a366-4523-8839-e65311db6bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:54.680433 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d2f58-a366-4523-8839-e65311db6bd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:54.681115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d2f58-a366-4523-8839-e65311db6bd1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:54.681416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d2f58-a366-4523-8839-e65311db6bd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:54.682808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d2f58-a366-4523-8839-e65311db6bd1 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:54.692188 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 163/328] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:54.692188 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d2f58-a366-4523-8839-e65311db6bd1 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:54.782808 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee559386-061d-48d5-a27b-c65ef051b5b9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:54.783640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:54.815179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d2f58-a366-4523-8839-e65311db6bd1 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:54.829494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faddb60-3d67-4826-bf44-91828db6c04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:54.830535 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faddb60-3d67-4826-bf44-91828db6c04a None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:54.831031 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faddb60-3d67-4826-bf44-91828db6c04a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:54.831511 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faddb60-3d67-4826-bf44-91828db6c04a None admin] PATH_INFO: `/v3/groups` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:54.832686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faddb60-3d67-4826-bf44-91828db6c04a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:54.835539 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faddb60-3d67-4826-bf44-91828db6c04a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:54.859252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb3444-8632-4e32-9698-db069971189c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:54.860189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3444-8632-4e32-9698-db069971189c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:54.860720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3444-8632-4e32-9698-db069971189c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:54.861209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3444-8632-4e32-9698-db069971189c None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:54.862356 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb3444-8632-4e32-9698-db069971189c None admin] RBAC: Adding query filter params (name=member) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:54.862667 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb3444-8632-4e32-9698-db069971189c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:54.864689 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb3444-8632-4e32-9698-db069971189c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:54.880179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 164/329] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:54.881733 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb3444-8632-4e32-9698-db069971189c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:54.951318 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 166/330] 10.222.0.70 () {60 vars in 1134 bytes} [Wed Aug 24 16:27:54 2022] POST /identity/v3/groups => generated 226 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:54.953032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faddb60-3d67-4826-bf44-91828db6c04a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:55.049399 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb3444-8632-4e32-9698-db069971189c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:55.087666 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:55.112298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faddb60-3d67-4826-bf44-91828db6c04a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:55.115101 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98090c4-5adc-4163-a76e-372614ce694b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:55.115857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98090c4-5adc-4163-a76e-372614ce694b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:55.116142 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98090c4-5adc-4163-a76e-372614ce694b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:55.116495 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98090c4-5adc-4163-a76e-372614ce694b None admin] PATH_INFO: `/v3/users/62d476ab9f20435b8ffb33994ef4b92f` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:55.118006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98090c4-5adc-4163-a76e-372614ce694b None admin] RBAC: Authorizing `identity:get_user(user_id=62d476ab9f20435b8ffb33994ef4b92f)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:55.120860 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98090c4-5adc-4163-a76e-372614ce694b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:55.123147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 165/331] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:54 2022] GET /identity/v3/users/62d476ab9f20435b8ffb33994ef4b92f => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:55.130615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98090c4-5adc-4163-a76e-372614ce694b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:55.151201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:55.165747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde0f4e-bc14-4a71-940f-13b41a85285a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:55.166817 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde0f4e-bc14-4a71-940f-13b41a85285a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:55.168443 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde0f4e-bc14-4a71-940f-13b41a85285a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:55.168443 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde0f4e-bc14-4a71-940f-13b41a85285a None admin] PATH_INFO: `/v3/users/03d13266d68141a3b919dae115eb51e9` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:55.169994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde0f4e-bc14-4a71-940f-13b41a85285a None admin] RBAC: Authorizing `identity:get_user(user_id=03d13266d68141a3b919dae115eb51e9)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:55.173019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde0f4e-bc14-4a71-940f-13b41a85285a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:55.176287 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 167/332] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:54 2022] GET /identity/v3/users/03d13266d68141a3b919dae115eb51e9 => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:55.182859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde0f4e-bc14-4a71-940f-13b41a85285a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:55.301151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98090c4-5adc-4163-a76e-372614ce694b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:55.346210 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022b92d-e714-4034-b35d-ef1f4b4245b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:55.347501 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022b92d-e714-4034-b35d-ef1f4b4245b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:55.348021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022b92d-e714-4034-b35d-ef1f4b4245b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:55.348884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022b92d-e714-4034-b35d-ef1f4b4245b0 None admin] PATH_INFO: `/v3/system/users/62d476ab9f20435b8ffb33994ef4b92f/roles/a915b705941d46538bd6a59aac287977` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:55.353388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022b92d-e714-4034-b35d-ef1f4b4245b0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=62d476ab9f20435b8ffb33994ef4b92f, role_id=a915b705941d46538bd6a59aac287977)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:55.359301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022b92d-e714-4034-b35d-ef1f4b4245b0 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:55.362797 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde0f4e-bc14-4a71-940f-13b41a85285a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:55.419254 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec537c0-94db-47f5-9dc2-bc4532990cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:55.420022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec537c0-94db-47f5-9dc2-bc4532990cff None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:55.420353 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec537c0-94db-47f5-9dc2-bc4532990cff None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:55.420668 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec537c0-94db-47f5-9dc2-bc4532990cff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:55.422539 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec537c0-94db-47f5-9dc2-bc4532990cff None admin] RBAC: Adding query filter params (role.id=c7b826398b4e466491d4723321aaa5f1, user.id=03d13266d68141a3b919dae115eb51e9, scope.system=all) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:55.422767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec537c0-94db-47f5-9dc2-bc4532990cff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:55.425593 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec537c0-94db-47f5-9dc2-bc4532990cff None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:55.431103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 166/333] 10.222.0.70 () {58 vars in 1334 bytes} [Wed Aug 24 16:27:55 2022] PUT /identity/v3/system/users/62d476ab9f20435b8ffb33994ef4b92f/roles/a915b705941d46538bd6a59aac287977 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:27:55.449134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 168/334] 10.222.0.70 () {56 vars in 1308 bytes} [Wed Aug 24 16:27:55 2022] GET /identity/v3/role_assignments?user.id=03d13266d68141a3b919dae115eb51e9&scope.system=all&role.id=c7b826398b4e466491d4723321aaa5f1 => generated 510 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:57.799040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27990273-ec9b-4d85-a2c8-6124013962dc None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:57.799802 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27990273-ec9b-4d85-a2c8-6124013962dc None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:57.799802 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27990273-ec9b-4d85-a2c8-6124013962dc None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:57.801213 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 167/335] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:57.817812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f827481-fb0d-4396-8afa-ffad9065bc86 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:57.818586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f827481-fb0d-4396-8afa-ffad9065bc86 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:57.818586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f827481-fb0d-4396-8afa-ffad9065bc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:57.849541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4f827481-fb0d-4396-8afa-ffad9065bc86 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:57.850772 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4f827481-fb0d-4396-8afa-ffad9065bc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:57.853054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:57.951212 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 169/336] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:58.572368 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27990273-ec9b-4d85-a2c8-6124013962dc None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:58.596050 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4042014-3b05-4fd3-8f87-2bd64a955d76 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:58.597178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4042014-3b05-4fd3-8f87-2bd64a955d76 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:58.598111 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4042014-3b05-4fd3-8f87-2bd64a955d76 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:58.603243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 170/337] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:58.610912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8482a-868c-4827-9dc1-5b5063d116b2 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:58.612032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8482a-868c-4827-9dc1-5b5063d116b2 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:58.612863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8482a-868c-4827-9dc1-5b5063d116b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:58.654596 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c5f8482a-868c-4827-9dc1-5b5063d116b2 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:58.663081 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c5f8482a-868c-4827-9dc1-5b5063d116b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:58.705782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:58.789854 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27990273-ec9b-4d85-a2c8-6124013962dc None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:58.872630 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 171/338] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:58.897310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a681a2-1bcb-4993-bb0d-f2661a9aeb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:58.898133 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a681a2-1bcb-4993-bb0d-f2661a9aeb43 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:58.898267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a681a2-1bcb-4993-bb0d-f2661a9aeb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:58.898624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a681a2-1bcb-4993-bb0d-f2661a9aeb43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:58.917574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a681a2-1bcb-4993-bb0d-f2661a9aeb43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-a6a681a2-1bcb-4993-bb0d-f2661a9aeb43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:58.924147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:58.925708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:58.927279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:58.927279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:58.927279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:58.927279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:58.927279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:58.927279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:58.927279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:58.927279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:58.927279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:58.927279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:58.927279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:27:58.927279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:27:58.927279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 168/339] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:58.938098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8482a-868c-4827-9dc1-5b5063d116b2 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:58.968669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f9aac-1fdc-4406-a538-f6c5c40df54f None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:58.969083 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f9aac-1fdc-4406-a538-f6c5c40df54f None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:58.969236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f9aac-1fdc-4406-a538-f6c5c40df54f None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:58.970845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 169/340] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:58.979777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1532eec-da69-4676-b694-0b5602d52025 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:58.979777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1532eec-da69-4676-b694-0b5602d52025 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:58.979777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1532eec-da69-4676-b694-0b5602d52025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:59.013510 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-c1532eec-da69-4676-b694-0b5602d52025 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:59.015256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c1532eec-da69-4676-b694-0b5602d52025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:59.153482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:59.243104 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 170/341] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:59.243791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8482a-868c-4827-9dc1-5b5063d116b2 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:59.245228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5028264-8fd0-4a55-9fac-91f2280792c7 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:59.245687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5028264-8fd0-4a55-9fac-91f2280792c7 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:59.246135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5028264-8fd0-4a55-9fac-91f2280792c7 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:59.248782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 171/342] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:27:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:59.272165 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f3677-f367-4f42-b3d5-9fe8fa1a547e None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:59.272625 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f3677-f367-4f42-b3d5-9fe8fa1a547e None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:59.273154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f3677-f367-4f42-b3d5-9fe8fa1a547e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:59.318078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-d61f3677-f367-4f42-b3d5-9fe8fa1a547e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:59.319961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d61f3677-f367-4f42-b3d5-9fe8fa1a547e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:59.330328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22197ff7-ddd4-4c42-af81-2dc795ef3625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:59.331863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22197ff7-ddd4-4c42-af81-2dc795ef3625 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:59.332660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22197ff7-ddd4-4c42-af81-2dc795ef3625 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:59.333388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22197ff7-ddd4-4c42-af81-2dc795ef3625 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:59.334799 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22197ff7-ddd4-4c42-af81-2dc795ef3625 None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:59.335607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22197ff7-ddd4-4c42-af81-2dc795ef3625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:59.338488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22197ff7-ddd4-4c42-af81-2dc795ef3625 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:59.357197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 172/343] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:59.359067 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22197ff7-ddd4-4c42-af81-2dc795ef3625 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:59.513908 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:59.568858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 172/344] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:59.570566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f3677-f367-4f42-b3d5-9fe8fa1a547e None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:59.667904 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22197ff7-ddd4-4c42-af81-2dc795ef3625 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:59.740989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:59.756011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288013c-a6f3-4a8a-a172-8ff69215eec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:59.756968 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288013c-a6f3-4a8a-a172-8ff69215eec3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:59.756968 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288013c-a6f3-4a8a-a172-8ff69215eec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:59.757217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288013c-a6f3-4a8a-a172-8ff69215eec3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:59.779974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288013c-a6f3-4a8a-a172-8ff69215eec3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-2288013c-a6f3-4a8a-a172-8ff69215eec3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:59.788445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:59.789396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:59.790184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:59.790184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:59.790184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:59.790184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:59.790184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:59.790184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:59.790184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:59.790184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:59.790184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:59.790184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:59.790184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:27:59.790184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:27:59.793940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 173/345] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 435 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:59.795059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288013c-a6f3-4a8a-a172-8ff69215eec3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:59.895785 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f3677-f367-4f42-b3d5-9fe8fa1a547e None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:59.975973 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f69d41-469a-4b79-9398-264cbdb2a018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:59.976779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f69d41-469a-4b79-9398-264cbdb2a018 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:59.976953 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f69d41-469a-4b79-9398-264cbdb2a018 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:59.977151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f69d41-469a-4b79-9398-264cbdb2a018 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:00.038146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288013c-a6f3-4a8a-a172-8ff69215eec3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:00.088821 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcfc39-35cc-4eb8-9812-d8f0ddca3352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:00.089473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcfc39-35cc-4eb8-9812-d8f0ddca3352 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:00.089752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcfc39-35cc-4eb8-9812-d8f0ddca3352 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:00.090027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcfc39-35cc-4eb8-9812-d8f0ddca3352 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:00.103335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcfc39-35cc-4eb8-9812-d8f0ddca3352 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-94bcfc39-35cc-4eb8-9812-d8f0ddca3352 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:00.109463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:00.111727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:00.113741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:00.113741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:00.113741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:00.113741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:00.113741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:00.113741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:00.113741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:00.113741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:00.113741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:00.113741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:00.113741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:28:00.113741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:00.113741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 174/346] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:00.113741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcfc39-35cc-4eb8-9812-d8f0ddca3352 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:00.140807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:00.146546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f69d41-469a-4b79-9398-264cbdb2a018 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:00.149551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f69d41-469a-4b79-9398-264cbdb2a018 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:00.175467 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:00.253054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcfc39-35cc-4eb8-9812-d8f0ddca3352 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:00.309975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc48888-7423-43fb-b1aa-3607cbf04f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:00.310524 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc48888-7423-43fb-b1aa-3607cbf04f7a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:00.311520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc48888-7423-43fb-b1aa-3607cbf04f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:00.311520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc48888-7423-43fb-b1aa-3607cbf04f7a None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:00.311956 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc48888-7423-43fb-b1aa-3607cbf04f7a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:00.312261 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc48888-7423-43fb-b1aa-3607cbf04f7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:00.313805 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc48888-7423-43fb-b1aa-3607cbf04f7a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:00.324063 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 175/347] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:00.325003 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc48888-7423-43fb-b1aa-3607cbf04f7a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:00.337236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 173/348] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 768 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:00.338172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f69d41-469a-4b79-9398-264cbdb2a018 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:00.486903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc48888-7423-43fb-b1aa-3607cbf04f7a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:00.503065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:00.504764 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:00.516780 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f69d41-469a-4b79-9398-264cbdb2a018 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:00.535533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ad1fc-aec5-4e91-a8b1-7af99fc4b4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:00.535533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ad1fc-aec5-4e91-a8b1-7af99fc4b4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:00.535533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ad1fc-aec5-4e91-a8b1-7af99fc4b4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:00.535533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ad1fc-aec5-4e91-a8b1-7af99fc4b4a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:00.555333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ad1fc-aec5-4e91-a8b1-7af99fc4b4a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-055ad1fc-aec5-4e91-a8b1-7af99fc4b4a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:00.564816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:00.566082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:00.569229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:00.569229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:00.569229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:00.569229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:00.569229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:00.569229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:00.569229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:00.569229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:00.569229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:00.569229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:00.569229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:28:00.569229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:00.569229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 176/349] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:00.570768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2533adf-4239-4789-a55f-31e696f73148 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:00.571131 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2533adf-4239-4789-a55f-31e696f73148 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:00.571515 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2533adf-4239-4789-a55f-31e696f73148 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:00.572933 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 177/350] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:00.574442 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a196e23-cebb-45a4-a495-465789a9f2b8 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:00.574726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a196e23-cebb-45a4-a495-465789a9f2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:00.574993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a196e23-cebb-45a4-a495-465789a9f2b8 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:00.578978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 178/351] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:00.579900 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a196e23-cebb-45a4-a495-465789a9f2b8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:00.582021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822c1ad-086b-49b8-ac1a-9577998cd6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:00.583103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822c1ad-086b-49b8-ac1a-9577998cd6cd None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:00.583395 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822c1ad-086b-49b8-ac1a-9577998cd6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:00.584632 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822c1ad-086b-49b8-ac1a-9577998cd6cd None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:00.585400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822c1ad-086b-49b8-ac1a-9577998cd6cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:00.585714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822c1ad-086b-49b8-ac1a-9577998cd6cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:00.587148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822c1ad-086b-49b8-ac1a-9577998cd6cd None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:00.598268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 174/352] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:28:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:00.599148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822c1ad-086b-49b8-ac1a-9577998cd6cd None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:00.731396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a196e23-cebb-45a4-a495-465789a9f2b8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:00.749885 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822c1ad-086b-49b8-ac1a-9577998cd6cd None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:00.782910 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d878a4-b507-464b-bec6-a9819f1eae09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:00.783563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d878a4-b507-464b-bec6-a9819f1eae09 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:00.783853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d878a4-b507-464b-bec6-a9819f1eae09 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:00.784132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d878a4-b507-464b-bec6-a9819f1eae09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:00.808254 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84a56c-e5f6-4829-bc17-e965577b435c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:00.809473 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84a56c-e5f6-4829-bc17-e965577b435c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:00.809938 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84a56c-e5f6-4829-bc17-e965577b435c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:00.810412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84a56c-e5f6-4829-bc17-e965577b435c None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:00.812735 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84a56c-e5f6-4829-bc17-e965577b435c None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:00.815641 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84a56c-e5f6-4829-bc17-e965577b435c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:00.818874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 175/353] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:00 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:00.820353 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84a56c-e5f6-4829-bc17-e965577b435c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:00.854455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:00.861459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:00.912393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d878a4-b507-464b-bec6-a9819f1eae09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:00.914889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d878a4-b507-464b-bec6-a9819f1eae09 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:00.970774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84a56c-e5f6-4829-bc17-e965577b435c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:01.024444 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a23dd1-16c7-4861-b10a-d848bda3f35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:01.025067 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a23dd1-16c7-4861-b10a-d848bda3f35e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:01.025245 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a23dd1-16c7-4861-b10a-d848bda3f35e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:01.025533 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a23dd1-16c7-4861-b10a-d848bda3f35e None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:01.026233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a23dd1-16c7-4861-b10a-d848bda3f35e None admin] RBAC: Adding query filter params (name=member) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:01.026544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a23dd1-16c7-4861-b10a-d848bda3f35e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:01.027963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a23dd1-16c7-4861-b10a-d848bda3f35e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:01.042435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 176/354] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:28:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:01.043802 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242157d0-d383-4976-8986-004b4b843f79 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:01.044179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242157d0-d383-4976-8986-004b4b843f79 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:01.044481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242157d0-d383-4976-8986-004b4b843f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:01.064423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 179/355] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:01.064972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-242157d0-d383-4976-8986-004b4b843f79 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:01.066056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23798870-f18f-4bc9-9ce0-0dcf4154bcb3 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:01.066159 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-242157d0-d383-4976-8986-004b4b843f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:01.066240 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23798870-f18f-4bc9-9ce0-0dcf4154bcb3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:01.066520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23798870-f18f-4bc9-9ce0-0dcf4154bcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:01.090017 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-23798870-f18f-4bc9-9ce0-0dcf4154bcb3 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:01.091253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-23798870-f18f-4bc9-9ce0-0dcf4154bcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:01.184539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 177/356] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:01.184539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242157d0-d383-4976-8986-004b4b843f79 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:01.194922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 180/357] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:01.196367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23798870-f18f-4bc9-9ce0-0dcf4154bcb3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:01.201110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:01.205710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:01.314200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242157d0-d383-4976-8986-004b4b843f79 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:01.355216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c0246-e8b8-4f0a-a743-37b56f668b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:01.356226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c0246-e8b8-4f0a-a743-37b56f668b54 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:01.356770 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c0246-e8b8-4f0a-a743-37b56f668b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:01.357247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c0246-e8b8-4f0a-a743-37b56f668b54 None admin] PATH_INFO: `/v3/users/62d476ab9f20435b8ffb33994ef4b92f` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:01.358837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c0246-e8b8-4f0a-a743-37b56f668b54 None admin] RBAC: Authorizing `identity:get_user(user_id=62d476ab9f20435b8ffb33994ef4b92f)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:01.360609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c0246-e8b8-4f0a-a743-37b56f668b54 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:01.362562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 178/358] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:28:01 2022] GET /identity/v3/users/62d476ab9f20435b8ffb33994ef4b92f => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:01.363978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c0246-e8b8-4f0a-a743-37b56f668b54 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:01.389963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23798870-f18f-4bc9-9ce0-0dcf4154bcb3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:01.454533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f2ad2-8c55-462f-ad6b-1d9e5ee8ab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:01.455269 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f2ad2-8c55-462f-ad6b-1d9e5ee8ab0e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:01.455956 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f2ad2-8c55-462f-ad6b-1d9e5ee8ab0e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:01.455956 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f2ad2-8c55-462f-ad6b-1d9e5ee8ab0e None admin] PATH_INFO: `/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:01.461968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:01.466487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f2ad2-8c55-462f-ad6b-1d9e5ee8ab0e None admin] RBAC: Authorizing `identity:get_group(group_id=f37e627622cc425dbf2e0b66f91d2ac2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:01.468580 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f2ad2-8c55-462f-ad6b-1d9e5ee8ab0e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:01.476255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 181/359] 10.222.0.70 () {56 vars in 1180 bytes} [Wed Aug 24 16:28:01 2022] GET /identity/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2 => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:01.476255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f2ad2-8c55-462f-ad6b-1d9e5ee8ab0e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:01.492298 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c0246-e8b8-4f0a-a743-37b56f668b54 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:01.536793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ea597-7bb2-4264-ab8c-231d2bcdb970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:01.537497 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ea597-7bb2-4264-ab8c-231d2bcdb970 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:01.537783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ea597-7bb2-4264-ab8c-231d2bcdb970 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:01.538059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ea597-7bb2-4264-ab8c-231d2bcdb970 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:01.566254 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:01.637153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f2ad2-8c55-462f-ad6b-1d9e5ee8ab0e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:01.651216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ea597-7bb2-4264-ab8c-231d2bcdb970 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:01.653066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ea597-7bb2-4264-ab8c-231d2bcdb970 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:01.683683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f7c4f-2e8f-445e-8134-166ea0a51f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:01.684344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f7c4f-2e8f-445e-8134-166ea0a51f2a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:01.684653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f7c4f-2e8f-445e-8134-166ea0a51f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:01.684923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f7c4f-2e8f-445e-8134-166ea0a51f2a None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:01.687170 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f7c4f-2e8f-445e-8134-166ea0a51f2a None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:01.688737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f7c4f-2e8f-445e-8134-166ea0a51f2a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:01.695085 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 182/360] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:01 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:01.695085 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f7c4f-2e8f-445e-8134-166ea0a51f2a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:01.778458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:01.805357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 179/361] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:01.805937 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ea597-7bb2-4264-ab8c-231d2bcdb970 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:01.842203 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f7c4f-2e8f-445e-8134-166ea0a51f2a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:01.897759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cdd6c-e3dd-4170-8a5c-7e28e517e851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:01.898332 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cdd6c-e3dd-4170-8a5c-7e28e517e851 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:01.898615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cdd6c-e3dd-4170-8a5c-7e28e517e851 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:01.898884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cdd6c-e3dd-4170-8a5c-7e28e517e851 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:01.899723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7cdd6c-e3dd-4170-8a5c-7e28e517e851 None admin] RBAC: Adding query filter params (role.id=a915b705941d46538bd6a59aac287977, user.id=62d476ab9f20435b8ffb33994ef4b92f, scope.system=all) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:01.900128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7cdd6c-e3dd-4170-8a5c-7e28e517e851 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:01.903659 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7cdd6c-e3dd-4170-8a5c-7e28e517e851 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:01.906161 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:01.924556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 183/362] 10.222.0.70 () {56 vars in 1308 bytes} [Wed Aug 24 16:28:01 2022] GET /identity/v3/role_assignments?user.id=62d476ab9f20435b8ffb33994ef4b92f&scope.system=all&role.id=a915b705941d46538bd6a59aac287977 => generated 510 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:01.925907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cdd6c-e3dd-4170-8a5c-7e28e517e851 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:01.938044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ea597-7bb2-4264-ab8c-231d2bcdb970 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:01.978211 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bee2c-db7a-4a6a-b85e-5678a4de6acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:01.978837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bee2c-db7a-4a6a-b85e-5678a4de6acf None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:01.979116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bee2c-db7a-4a6a-b85e-5678a4de6acf None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:01.979388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bee2c-db7a-4a6a-b85e-5678a4de6acf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:01.980149 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bee2c-db7a-4a6a-b85e-5678a4de6acf None admin] RBAC: Adding query filter params (group.id=f37e627622cc425dbf2e0b66f91d2ac2, role.id=c7b826398b4e466491d4723321aaa5f1, scope.project.id=41be92ad8eba49128ba42ffcedab9caf) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:01.980483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bee2c-db7a-4a6a-b85e-5678a4de6acf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:01.981981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bee2c-db7a-4a6a-b85e-5678a4de6acf None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:01.990534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 180/363] 10.222.0.70 () {56 vars in 1376 bytes} [Wed Aug 24 16:28:01 2022] GET /identity/v3/role_assignments?group.id=f37e627622cc425dbf2e0b66f91d2ac2&scope.project.id=41be92ad8eba49128ba42ffcedab9caf&role.id=c7b826398b4e466491d4723321aaa5f1 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:01.991445 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bee2c-db7a-4a6a-b85e-5678a4de6acf None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:02.063812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:02.083321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cdd6c-e3dd-4170-8a5c-7e28e517e851 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:02.130464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a39ae-e6ea-47cf-a584-9eca41283c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:02.131590 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a39ae-e6ea-47cf-a584-9eca41283c5f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:02.132155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a39ae-e6ea-47cf-a584-9eca41283c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:02.132699 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a39ae-e6ea-47cf-a584-9eca41283c5f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:02.136438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bee2c-db7a-4a6a-b85e-5678a4de6acf None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:02.147061 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a39ae-e6ea-47cf-a584-9eca41283c5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-ef3a39ae-e6ea-47cf-a584-9eca41283c5f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:02.152974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:02.153902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:02.154734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:02.154734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:02.154734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:02.154734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:02.154734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:02.154734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:02.154734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:02.154734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:02.154734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:02.154734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:02.154734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:28:02.154734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:02.157948 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 184/364] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:28:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:02.159153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a39ae-e6ea-47cf-a584-9eca41283c5f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:02.179975 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef8f93-043c-406e-b431-0e4c41b28253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:02.180749 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef8f93-043c-406e-b431-0e4c41b28253 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:02.180964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef8f93-043c-406e-b431-0e4c41b28253 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:02.181238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef8f93-043c-406e-b431-0e4c41b28253 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:02.202594 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef8f93-043c-406e-b431-0e4c41b28253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-40ef8f93-043c-406e-b431-0e4c41b28253 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:02.210155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:02.211141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:02.211942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:02.211942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:02.211942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:02.211942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:02.211942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:02.211942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:02.211942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:02.211942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:02.211942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:02.211942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:02.211942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:02.211942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:02.215382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 181/365] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:02.216667 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef8f93-043c-406e-b431-0e4c41b28253 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:02.232867 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:02.300892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a39ae-e6ea-47cf-a584-9eca41283c5f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:02.345280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab371595-e75c-462d-8987-6e714397296d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:02.346111 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab371595-e75c-462d-8987-6e714397296d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:02.346396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab371595-e75c-462d-8987-6e714397296d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:02.346726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab371595-e75c-462d-8987-6e714397296d None admin] PATH_INFO: `/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:02.347963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab371595-e75c-462d-8987-6e714397296d None admin] RBAC: Authorizing `identity:get_group(group_id=f37e627622cc425dbf2e0b66f91d2ac2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:02.349520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab371595-e75c-462d-8987-6e714397296d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:02.351334 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 185/366] 10.222.0.70 () {56 vars in 1180 bytes} [Wed Aug 24 16:28:02 2022] GET /identity/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:02.352144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab371595-e75c-462d-8987-6e714397296d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:02.387836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef8f93-043c-406e-b431-0e4c41b28253 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:02.398867 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:02.439390 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0063a-dbe2-4b50-ab05-4d766d68d926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:02.440313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0063a-dbe2-4b50-ab05-4d766d68d926 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:02.440772 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0063a-dbe2-4b50-ab05-4d766d68d926 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:02.441190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0063a-dbe2-4b50-ab05-4d766d68d926 None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:02.442660 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0063a-dbe2-4b50-ab05-4d766d68d926 None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:02.444277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0063a-dbe2-4b50-ab05-4d766d68d926 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:02.446267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 182/367] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:02 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:02.446878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0063a-dbe2-4b50-ab05-4d766d68d926 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:02.478688 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab371595-e75c-462d-8987-6e714397296d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:02.502314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:02.522277 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe8a79-9c7d-4cc0-b63c-56e93ddf43bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:02.523701 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe8a79-9c7d-4cc0-b63c-56e93ddf43bd None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:02.524219 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe8a79-9c7d-4cc0-b63c-56e93ddf43bd None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:02.524686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe8a79-9c7d-4cc0-b63c-56e93ddf43bd None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:02.525399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe8a79-9c7d-4cc0-b63c-56e93ddf43bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:02.525707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe8a79-9c7d-4cc0-b63c-56e93ddf43bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:02.527122 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe8a79-9c7d-4cc0-b63c-56e93ddf43bd None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:02.540703 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 186/368] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:02.541596 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe8a79-9c7d-4cc0-b63c-56e93ddf43bd None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:02.612281 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0063a-dbe2-4b50-ab05-4d766d68d926 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:02.655735 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fb9b7-df63-4b57-998a-6d6a226bd364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:02.656633 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fb9b7-df63-4b57-998a-6d6a226bd364 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:02.657051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fb9b7-df63-4b57-998a-6d6a226bd364 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:02.657469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fb9b7-df63-4b57-998a-6d6a226bd364 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:02.658505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fb9b7-df63-4b57-998a-6d6a226bd364 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:02.660028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fb9b7-df63-4b57-998a-6d6a226bd364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:02.661061 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fb9b7-df63-4b57-998a-6d6a226bd364 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:02.671697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 183/369] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:02.676440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fb9b7-df63-4b57-998a-6d6a226bd364 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:02.707853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe8a79-9c7d-4cc0-b63c-56e93ddf43bd None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:02.727890 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:02.748463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdbc429-094a-4f4d-bb1f-901cd90dd1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:02.749112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbc429-094a-4f4d-bb1f-901cd90dd1d2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:02.749396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbc429-094a-4f4d-bb1f-901cd90dd1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:02.749667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbc429-094a-4f4d-bb1f-901cd90dd1d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:02.750453 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdbc429-094a-4f4d-bb1f-901cd90dd1d2 None admin] RBAC: Adding query filter params (group.id=f37e627622cc425dbf2e0b66f91d2ac2, role.id=003cfd128ecf4c238197991ad36f0c9a, scope.project.id=41be92ad8eba49128ba42ffcedab9caf) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:02.750764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdbc429-094a-4f4d-bb1f-901cd90dd1d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:02.752201 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdbc429-094a-4f4d-bb1f-901cd90dd1d2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:02.768432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 187/370] 10.222.0.70 () {56 vars in 1376 bytes} [Wed Aug 24 16:28:02 2022] GET /identity/v3/role_assignments?group.id=f37e627622cc425dbf2e0b66f91d2ac2&scope.project.id=41be92ad8eba49128ba42ffcedab9caf&role.id=003cfd128ecf4c238197991ad36f0c9a => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:02.768432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdbc429-094a-4f4d-bb1f-901cd90dd1d2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:02.831318 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fb9b7-df63-4b57-998a-6d6a226bd364 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:02.844550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:02.876548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af67aac-945e-4538-993a-48a365e551b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:02.877313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af67aac-945e-4538-993a-48a365e551b5 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:02.877596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af67aac-945e-4538-993a-48a365e551b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:02.877931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af67aac-945e-4538-993a-48a365e551b5 None admin] PATH_INFO: `/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:02.879326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af67aac-945e-4538-993a-48a365e551b5 None admin] RBAC: Authorizing `identity:get_group(group_id=f37e627622cc425dbf2e0b66f91d2ac2)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:02.880835 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af67aac-945e-4538-993a-48a365e551b5 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:02.882818 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 184/371] 10.222.0.70 () {56 vars in 1180 bytes} [Wed Aug 24 16:28:02 2022] GET /identity/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:02.883676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af67aac-945e-4538-993a-48a365e551b5 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:02.929917 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdbc429-094a-4f4d-bb1f-901cd90dd1d2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:02.975399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e74cfd-94b7-4969-8a20-70c0f63dd5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:02.976355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e74cfd-94b7-4969-8a20-70c0f63dd5fe None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:02.976961 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e74cfd-94b7-4969-8a20-70c0f63dd5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:02.977434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e74cfd-94b7-4969-8a20-70c0f63dd5fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:03.041829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af67aac-945e-4538-993a-48a365e551b5 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:03.053049 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e74cfd-94b7-4969-8a20-70c0f63dd5fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:03.054523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e74cfd-94b7-4969-8a20-70c0f63dd5fe None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:03.054984 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:03.086075 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a3dd6-a9d9-48a8-81a7-27a24a4aaf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:03.087616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a3dd6-a9d9-48a8-81a7-27a24a4aaf5a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:03.088307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a3dd6-a9d9-48a8-81a7-27a24a4aaf5a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:03.089236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a3dd6-a9d9-48a8-81a7-27a24a4aaf5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:03.107850 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:03.169004 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 188/372] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:03.169971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e74cfd-94b7-4969-8a20-70c0f63dd5fe None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:03.186007 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a3dd6-a9d9-48a8-81a7-27a24a4aaf5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:03.189145 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a3dd6-a9d9-48a8-81a7-27a24a4aaf5a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:03.291496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e74cfd-94b7-4969-8a20-70c0f63dd5fe None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:03.327658 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 185/373] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:03.328905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a3dd6-a9d9-48a8-81a7-27a24a4aaf5a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:03.333379 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:03.337556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f576959-2e3d-4bea-8953-2b462059876e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:03.338289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f576959-2e3d-4bea-8953-2b462059876e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:03.339071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f576959-2e3d-4bea-8953-2b462059876e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:03.339071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f576959-2e3d-4bea-8953-2b462059876e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:03.339552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f576959-2e3d-4bea-8953-2b462059876e None admin] RBAC: Adding query filter params (group.id=f37e627622cc425dbf2e0b66f91d2ac2, role.id=c7b826398b4e466491d4723321aaa5f1, scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:03.339857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f576959-2e3d-4bea-8953-2b462059876e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:03.341343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f576959-2e3d-4bea-8953-2b462059876e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:03.353433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 189/374] 10.222.0.70 () {56 vars in 1376 bytes} [Wed Aug 24 16:28:03 2022] GET /identity/v3/role_assignments?group.id=f37e627622cc425dbf2e0b66f91d2ac2&scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15&role.id=c7b826398b4e466491d4723321aaa5f1 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:03.353433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f576959-2e3d-4bea-8953-2b462059876e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:03.377253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:03.532619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f576959-2e3d-4bea-8953-2b462059876e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:03.534654 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a3dd6-a9d9-48a8-81a7-27a24a4aaf5a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:03.591368 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c46c8-c58f-4080-99fa-47173eee0ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:03.592029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c46c8-c58f-4080-99fa-47173eee0ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:03.592491 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c46c8-c58f-4080-99fa-47173eee0ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:03.592749 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c46c8-c58f-4080-99fa-47173eee0ab5 None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:03.595019 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c46c8-c58f-4080-99fa-47173eee0ab5 None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:03.596767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c46c8-c58f-4080-99fa-47173eee0ab5 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:03.598841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 186/375] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:03 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:03.601918 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c7a3f-bea6-4ca7-a5c5-da8e1aaa21b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:03.602566 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c7a3f-bea6-4ca7-a5c5-da8e1aaa21b9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:03.603106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c7a3f-bea6-4ca7-a5c5-da8e1aaa21b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:03.603275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c7a3f-bea6-4ca7-a5c5-da8e1aaa21b9 None admin] PATH_INFO: `/v3/projects/8f3b91fa60b4447a866a572209ebc37f` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:03.604455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c7a3f-bea6-4ca7-a5c5-da8e1aaa21b9 None admin] RBAC: Authorizing `identity:get_project(project_id=8f3b91fa60b4447a866a572209ebc37f)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:03.604752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c46c8-c58f-4080-99fa-47173eee0ab5 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:03.605968 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c7a3f-bea6-4ca7-a5c5-da8e1aaa21b9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:03.612580 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 190/376] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:03 2022] GET /identity/v3/projects/8f3b91fa60b4447a866a572209ebc37f => generated 344 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:03.614387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c7a3f-bea6-4ca7-a5c5-da8e1aaa21b9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:03.687424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:03.741846 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c46c8-c58f-4080-99fa-47173eee0ab5 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:03.780987 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a2123-8790-43ba-82e6-fbc27c22ce40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:03.781984 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a2123-8790-43ba-82e6-fbc27c22ce40 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:03.782459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a2123-8790-43ba-82e6-fbc27c22ce40 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:03.782952 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a2123-8790-43ba-82e6-fbc27c22ce40 None admin] PATH_INFO: `/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:03.788877 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:03.789334 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a2123-8790-43ba-82e6-fbc27c22ce40 None admin] RBAC: Authorizing `identity:get_group(group_id=f37e627622cc425dbf2e0b66f91d2ac2)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:03.790805 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a2123-8790-43ba-82e6-fbc27c22ce40 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:03.792690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 187/377] 10.222.0.70 () {56 vars in 1180 bytes} [Wed Aug 24 16:28:03 2022] GET /identity/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:03.800982 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a2123-8790-43ba-82e6-fbc27c22ce40 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:03.836826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c7a3f-bea6-4ca7-a5c5-da8e1aaa21b9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:03.914681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f23e81-c8e9-4a07-80d4-8617769ac6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:03.915343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f23e81-c8e9-4a07-80d4-8617769ac6bf None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:03.915631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f23e81-c8e9-4a07-80d4-8617769ac6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:03.915910 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f23e81-c8e9-4a07-80d4-8617769ac6bf None admin] PATH_INFO: `/v3/groups/9687a8d318304e8c8dba3eac7d1eba4b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:03.926870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f23e81-c8e9-4a07-80d4-8617769ac6bf None admin] RBAC: Authorizing `identity:get_group(group_id=9687a8d318304e8c8dba3eac7d1eba4b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:03.928377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f23e81-c8e9-4a07-80d4-8617769ac6bf None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:03.930304 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 191/378] 10.222.0.70 () {56 vars in 1180 bytes} [Wed Aug 24 16:28:03 2022] GET /identity/v3/groups/9687a8d318304e8c8dba3eac7d1eba4b => generated 229 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:28:03.938644 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f23e81-c8e9-4a07-80d4-8617769ac6bf None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:04.010390 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a2123-8790-43ba-82e6-fbc27c22ce40 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:04.064315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562b8b4-f82c-471e-b21e-7ec98dea95bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:04.064891 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562b8b4-f82c-471e-b21e-7ec98dea95bf None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:04.065175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562b8b4-f82c-471e-b21e-7ec98dea95bf None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:04.065453 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562b8b4-f82c-471e-b21e-7ec98dea95bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:04.066202 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562b8b4-f82c-471e-b21e-7ec98dea95bf None admin] RBAC: Adding query filter params (group.id=f37e627622cc425dbf2e0b66f91d2ac2, role.id=003cfd128ecf4c238197991ad36f0c9a, scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:04.066512 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562b8b4-f82c-471e-b21e-7ec98dea95bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:04.068027 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562b8b4-f82c-471e-b21e-7ec98dea95bf None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:04.084771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 188/379] 10.222.0.70 () {56 vars in 1376 bytes} [Wed Aug 24 16:28:03 2022] GET /identity/v3/role_assignments?group.id=f37e627622cc425dbf2e0b66f91d2ac2&scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15&role.id=003cfd128ecf4c238197991ad36f0c9a => generated 260 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:28:04.120704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f23e81-c8e9-4a07-80d4-8617769ac6bf None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:04.168290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702645cc-dbc0-4750-9aeb-ce313cc7df0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:04.168970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702645cc-dbc0-4750-9aeb-ce313cc7df0b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:04.169257 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702645cc-dbc0-4750-9aeb-ce313cc7df0b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:04.169534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702645cc-dbc0-4750-9aeb-ce313cc7df0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:04.170319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702645cc-dbc0-4750-9aeb-ce313cc7df0b None admin] RBAC: Adding query filter params (group.id=9687a8d318304e8c8dba3eac7d1eba4b, role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=8f3b91fa60b4447a866a572209ebc37f) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:04.171197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702645cc-dbc0-4750-9aeb-ce313cc7df0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:04.172847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702645cc-dbc0-4750-9aeb-ce313cc7df0b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:04.179376 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 192/380] 10.222.0.70 () {56 vars in 1376 bytes} [Wed Aug 24 16:28:03 2022] GET /identity/v3/role_assignments?group.id=9687a8d318304e8c8dba3eac7d1eba4b&scope.project.id=8f3b91fa60b4447a866a572209ebc37f&role.id=e7f48343fc1542979ab4459865109e30 => generated 260 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:04.334234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75ee4b-3023-4eef-b37e-7b6b97598ced None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:04.334829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75ee4b-3023-4eef-b37e-7b6b97598ced None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:04.334829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75ee4b-3023-4eef-b37e-7b6b97598ced None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:04.336494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 189/381] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:04.343239 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7dbff5-fcf4-448b-8578-2f0cdaf2c7b2 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:04.343658 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7dbff5-fcf4-448b-8578-2f0cdaf2c7b2 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:04.343900 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7dbff5-fcf4-448b-8578-2f0cdaf2c7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:04.363448 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:04.377721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5a7dbff5-fcf4-448b-8578-2f0cdaf2c7b2 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:04.379003 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5a7dbff5-fcf4-448b-8578-2f0cdaf2c7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:04.473058 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 193/382] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:28:04.959562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75ee4b-3023-4eef-b37e-7b6b97598ced None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:04.965941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:04.990224 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c30b2-fd3b-4ac8-a613-5c39f0af08cb None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:04.990641 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c30b2-fd3b-4ac8-a613-5c39f0af08cb None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:04.990939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c30b2-fd3b-4ac8-a613-5c39f0af08cb None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:04.992801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 194/383] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:05.000232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62021f-d516-4521-89af-255dea0736a6 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:05.005764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62021f-d516-4521-89af-255dea0736a6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:05.005764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62021f-d516-4521-89af-255dea0736a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:05.036701 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ee62021f-d516-4521-89af-255dea0736a6 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:05.040060 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ee62021f-d516-4521-89af-255dea0736a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:05.148431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75ee4b-3023-4eef-b37e-7b6b97598ced None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:05.197563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 195/384] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:05.218283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16b246-13b4-4715-9d04-c57f85bf1434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:05.218867 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16b246-13b4-4715-9d04-c57f85bf1434 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:05.219149 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16b246-13b4-4715-9d04-c57f85bf1434 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:05.219425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16b246-13b4-4715-9d04-c57f85bf1434 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:05.236145 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16b246-13b4-4715-9d04-c57f85bf1434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-7c16b246-13b4-4715-9d04-c57f85bf1434 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:05.241525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:05.242494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:05.243297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:05.243297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:05.243297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:05.243297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:05.243297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:05.243297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:05.243297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:05.243297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:05.243297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:05.243297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:05.243297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:28:05.243297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:05.243297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 190/385] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:05.251946 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62021f-d516-4521-89af-255dea0736a6 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:05.395459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:05.503200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62021f-d516-4521-89af-255dea0736a6 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:05.591128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb610a-5946-4366-a054-25662b1edfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:05.591747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb610a-5946-4366-a054-25662b1edfcd None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:05.592025 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb610a-5946-4366-a054-25662b1edfcd None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:05.592836 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb610a-5946-4366-a054-25662b1edfcd None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:05.593621 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb610a-5946-4366-a054-25662b1edfcd None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:05.593922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb610a-5946-4366-a054-25662b1edfcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:05.595286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb610a-5946-4366-a054-25662b1edfcd None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:05.609793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16b246-13b4-4715-9d04-c57f85bf1434 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:05.616813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 196/386] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:28:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:05.626070 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb610a-5946-4366-a054-25662b1edfcd None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:05.741448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:05.809423 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16b246-13b4-4715-9d04-c57f85bf1434 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:05.848486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aaa02-7da6-4c95-9b3a-050843198299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:05.849350 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aaa02-7da6-4c95-9b3a-050843198299 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:05.849697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aaa02-7da6-4c95-9b3a-050843198299 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:05.850039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aaa02-7da6-4c95-9b3a-050843198299 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:05.862652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67aaa02-7da6-4c95-9b3a-050843198299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-f67aaa02-7da6-4c95-9b3a-050843198299 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:05.868576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:05.869583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:05.870407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:05.870407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:05.870407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:05.870407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:05.870407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:05.870407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:05.870407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:05.870407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:05.870407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:05.870407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:05.870407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:28:05.870407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:05.870407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 191/387] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:28:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:05.876668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aaa02-7da6-4c95-9b3a-050843198299 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:05.882562 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb610a-5946-4366-a054-25662b1edfcd None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:05.965766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:05.975495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df8ea2-bfcd-44de-b8da-ac426fc09b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:05.976325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df8ea2-bfcd-44de-b8da-ac426fc09b41 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:05.979502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df8ea2-bfcd-44de-b8da-ac426fc09b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:05.979502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df8ea2-bfcd-44de-b8da-ac426fc09b41 None admin] PATH_INFO: `/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:05.979502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df8ea2-bfcd-44de-b8da-ac426fc09b41 None admin] RBAC: Authorizing `identity:get_group(group_id=f37e627622cc425dbf2e0b66f91d2ac2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:05.980459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df8ea2-bfcd-44de-b8da-ac426fc09b41 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:05.982388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 197/388] 10.222.0.70 () {56 vars in 1180 bytes} [Wed Aug 24 16:28:05 2022] GET /identity/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2 => generated 226 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:05.983721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f7a1b-2ca7-44b0-967b-faefd3dccf7c None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:05.984003 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f7a1b-2ca7-44b0-967b-faefd3dccf7c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:05.984278 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f7a1b-2ca7-44b0-967b-faefd3dccf7c None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:05.986266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 198/389] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:05.992346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f7a1b-2ca7-44b0-967b-faefd3dccf7c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:06.034934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aaa02-7da6-4c95-9b3a-050843198299 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:06.078038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:06.098501 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee75e2-7d36-4f8d-8798-f44b0d3f847e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:06.099822 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee75e2-7d36-4f8d-8798-f44b0d3f847e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:06.100550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee75e2-7d36-4f8d-8798-f44b0d3f847e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:06.101351 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee75e2-7d36-4f8d-8798-f44b0d3f847e None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:06.102716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee75e2-7d36-4f8d-8798-f44b0d3f847e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:06.103487 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee75e2-7d36-4f8d-8798-f44b0d3f847e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:06.106365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee75e2-7d36-4f8d-8798-f44b0d3f847e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:06.125205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 192/390] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:06.127731 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb2c67-01d2-4708-9661-a7e0eebcad81 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:06.128549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb2c67-01d2-4708-9661-a7e0eebcad81 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:06.129248 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb2c67-01d2-4708-9661-a7e0eebcad81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:06.160164 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-b0eb2c67-01d2-4708-9661-a7e0eebcad81 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:06.163009 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b0eb2c67-01d2-4708-9661-a7e0eebcad81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:06.199599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f7a1b-2ca7-44b0-967b-faefd3dccf7c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:06.265375 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa0064-4890-47fa-a040-597402f0ec1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:06.266681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa0064-4890-47fa-a040-597402f0ec1d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:06.267195 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa0064-4890-47fa-a040-597402f0ec1d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:06.267655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa0064-4890-47fa-a040-597402f0ec1d None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:06.269707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa0064-4890-47fa-a040-597402f0ec1d None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:06.272427 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa0064-4890-47fa-a040-597402f0ec1d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:06.275938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 199/391] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:05 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:06.277704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa0064-4890-47fa-a040-597402f0ec1d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:06.290382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 193/392] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:06.292333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb2c67-01d2-4708-9661-a7e0eebcad81 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:06.464849 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:06.485710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb2c67-01d2-4708-9661-a7e0eebcad81 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:06.564736 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:06.585313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e8849-a748-46ed-bab5-bd988220f17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:06.587017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e8849-a748-46ed-bab5-bd988220f17d None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:06.587686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e8849-a748-46ed-bab5-bd988220f17d None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:06.588420 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e8849-a748-46ed-bab5-bd988220f17d None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf/groups/f37e627622cc425dbf2e0b66f91d2ac2/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:06.589498 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa0064-4890-47fa-a040-597402f0ec1d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:06.592035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e8849-a748-46ed-bab5-bd988220f17d None admin] RBAC: Authorizing `identity:create_grant(project_id=41be92ad8eba49128ba42ffcedab9caf, group_id=f37e627622cc425dbf2e0b66f91d2ac2, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:06.596176 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e8849-a748-46ed-bab5-bd988220f17d None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:06.637332 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73662894-1a01-4b73-b017-5e1fd5a4d2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:06.637962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73662894-1a01-4b73-b017-5e1fd5a4d2ab None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:06.638232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73662894-1a01-4b73-b017-5e1fd5a4d2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:06.638501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73662894-1a01-4b73-b017-5e1fd5a4d2ab None admin] PATH_INFO: `/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:06.639684 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73662894-1a01-4b73-b017-5e1fd5a4d2ab None admin] RBAC: Authorizing `identity:get_group(group_id=f37e627622cc425dbf2e0b66f91d2ac2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:06.641184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73662894-1a01-4b73-b017-5e1fd5a4d2ab None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:06.643516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 200/393] 10.222.0.70 () {56 vars in 1180 bytes} [Wed Aug 24 16:28:06 2022] GET /identity/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2 => generated 226 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:06.648445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c9192-a5eb-4c60-b13a-033255888a3f None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:06.648445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c9192-a5eb-4c60-b13a-033255888a3f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:06.648445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c9192-a5eb-4c60-b13a-033255888a3f None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:06.648445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 201/394] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:06.653874 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0e3fa-cf21-4072-a5c6-174f3a85aad7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:06.654152 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0e3fa-cf21-4072-a5c6-174f3a85aad7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:06.654427 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0e3fa-cf21-4072-a5c6-174f3a85aad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:06.675270 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-06e0e3fa-cf21-4072-a5c6-174f3a85aad7 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:06.676495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-06e0e3fa-cf21-4072-a5c6-174f3a85aad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:06.694505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 194/395] 10.222.0.70 () {58 vars in 1442 bytes} [Wed Aug 24 16:28:06 2022] PUT /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf/groups/f37e627622cc425dbf2e0b66f91d2ac2/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:28:06.697600 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e8849-a748-46ed-bab5-bd988220f17d None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:06.810292 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 202/396] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:06.813239 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0e3fa-cf21-4072-a5c6-174f3a85aad7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:06.848483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e8849-a748-46ed-bab5-bd988220f17d None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:06.889590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050f85e-b6ab-48c0-9465-8e1ee5df6f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:06.890710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050f85e-b6ab-48c0-9465-8e1ee5df6f03 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:06.891209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050f85e-b6ab-48c0-9465-8e1ee5df6f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:06.891984 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050f85e-b6ab-48c0-9465-8e1ee5df6f03 None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:06.893609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050f85e-b6ab-48c0-9465-8e1ee5df6f03 None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:06.895350 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050f85e-b6ab-48c0-9465-8e1ee5df6f03 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:06.897730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 195/397] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:06 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:06.899899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:06.900477 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1cd7c-08fb-4a5c-90a3-c10d8cdbfa36 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:06.901002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1cd7c-08fb-4a5c-90a3-c10d8cdbfa36 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:06.901484 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1cd7c-08fb-4a5c-90a3-c10d8cdbfa36 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:06.907958 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 196/398] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:06.909220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1cd7c-08fb-4a5c-90a3-c10d8cdbfa36 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:06.949610 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0e3fa-cf21-4072-a5c6-174f3a85aad7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:06.965889 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:07.006559 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078f876-a5dc-49c7-9d3c-c06f0052876d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:07.007270 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078f876-a5dc-49c7-9d3c-c06f0052876d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:07.007545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078f876-a5dc-49c7-9d3c-c06f0052876d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:07.007863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078f876-a5dc-49c7-9d3c-c06f0052876d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:07.024620 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078f876-a5dc-49c7-9d3c-c06f0052876d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-6078f876-a5dc-49c7-9d3c-c06f0052876d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:07.031066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:07.031974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:07.032802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:07.032802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:07.032802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:07.032802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:07.032802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:07.032802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:07.032802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:07.032802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:07.032802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:07.032802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:07.032802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:28:07.032802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:07.032802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 203/399] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:07.038129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70a11a-8ccd-4705-ad69-7a41a92e20ad None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:07.038129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70a11a-8ccd-4705-ad69-7a41a92e20ad None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:07.038129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70a11a-8ccd-4705-ad69-7a41a92e20ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:07.058862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0e70a11a-8ccd-4705-ad69-7a41a92e20ad None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:07.060367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0e70a11a-8ccd-4705-ad69-7a41a92e20ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:07.067636 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1cd7c-08fb-4a5c-90a3-c10d8cdbfa36 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:07.120691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4051645-320b-45de-94ac-0dc2b85442a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:07.121564 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4051645-320b-45de-94ac-0dc2b85442a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:07.121962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4051645-320b-45de-94ac-0dc2b85442a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:07.122286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4051645-320b-45de-94ac-0dc2b85442a7 None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf/groups/f37e627622cc425dbf2e0b66f91d2ac2/roles/003cfd128ecf4c238197991ad36f0c9a` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:07.124898 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4051645-320b-45de-94ac-0dc2b85442a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=41be92ad8eba49128ba42ffcedab9caf, group_id=f37e627622cc425dbf2e0b66f91d2ac2, role_id=003cfd128ecf4c238197991ad36f0c9a)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:07.126434 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4051645-320b-45de-94ac-0dc2b85442a7 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:07.152556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 204/400] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:07.153420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70a11a-8ccd-4705-ad69-7a41a92e20ad None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:07.216523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:07.255060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 197/401] 10.222.0.70 () {58 vars in 1442 bytes} [Wed Aug 24 16:28:06 2022] PUT /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf/groups/f37e627622cc425dbf2e0b66f91d2ac2/roles/003cfd128ecf4c238197991ad36f0c9a => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:28:07.255060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4051645-320b-45de-94ac-0dc2b85442a7 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:07.330312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70a11a-8ccd-4705-ad69-7a41a92e20ad None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:07.413638 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2ef6c-80b3-4601-87cf-05031c0595e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:07.414323 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2ef6c-80b3-4601-87cf-05031c0595e4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:07.414605 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2ef6c-80b3-4601-87cf-05031c0595e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:07.414873 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2ef6c-80b3-4601-87cf-05031c0595e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:07.416898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2ef6c-80b3-4601-87cf-05031c0595e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:07.417207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2ef6c-80b3-4601-87cf-05031c0595e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:07.419004 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2ef6c-80b3-4601-87cf-05031c0595e4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:07.437230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 205/402] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:28:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:07.438813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2ef6c-80b3-4601-87cf-05031c0595e4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:07.468481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4051645-320b-45de-94ac-0dc2b85442a7 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:07.472243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:07.518205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0f481-3170-4ec6-9435-afed336f033a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:07.519452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0f481-3170-4ec6-9435-afed336f033a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:07.520850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0f481-3170-4ec6-9435-afed336f033a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:07.521410 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0f481-3170-4ec6-9435-afed336f033a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:07.537749 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0f481-3170-4ec6-9435-afed336f033a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-9ff0f481-3170-4ec6-9435-afed336f033a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:07.544371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:07.545313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:07.546252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:07.546252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:07.546252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:07.546252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:07.546252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:07.546252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:07.546252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:07.546252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:07.546252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:07.546252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:07.546252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:07.546252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:07.549937 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 198/403] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:07.551464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0f481-3170-4ec6-9435-afed336f033a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:07.569176 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2ef6c-80b3-4601-87cf-05031c0595e4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:07.586008 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:07.614617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33519692-c946-4e6b-8000-a31929fafae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:07.615260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33519692-c946-4e6b-8000-a31929fafae8 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:07.615537 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33519692-c946-4e6b-8000-a31929fafae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:07.615810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33519692-c946-4e6b-8000-a31929fafae8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:07.629719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33519692-c946-4e6b-8000-a31929fafae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-33519692-c946-4e6b-8000-a31929fafae8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:07.634750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:07.635686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:07.641816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:07.641816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:07.641816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:07.641816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:07.641816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:07.641816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:07.641816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:07.641816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:07.641816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:07.641816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:07.641816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:28:07.641816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:07.641816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 206/404] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:28:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:07.641816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33519692-c946-4e6b-8000-a31929fafae8 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:07.709106 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0f481-3170-4ec6-9435-afed336f033a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:07.751700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339104ba-549f-401e-bdef-5242216e4e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:07.752489 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339104ba-549f-401e-bdef-5242216e4e83 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:07.753032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339104ba-549f-401e-bdef-5242216e4e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:07.753032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339104ba-549f-401e-bdef-5242216e4e83 None admin] PATH_INFO: `/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:07.754281 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339104ba-549f-401e-bdef-5242216e4e83 None admin] RBAC: Authorizing `identity:get_group(group_id=f37e627622cc425dbf2e0b66f91d2ac2)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:07.756062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339104ba-549f-401e-bdef-5242216e4e83 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:07.763130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 199/405] 10.222.0.70 () {56 vars in 1180 bytes} [Wed Aug 24 16:28:07 2022] GET /identity/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:07.764960 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339104ba-549f-401e-bdef-5242216e4e83 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:07.785860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33519692-c946-4e6b-8000-a31929fafae8 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:07.803118 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:07.840174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fb169-3d32-491c-b999-9133cc46ce09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:07.840792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fb169-3d32-491c-b999-9133cc46ce09 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:07.841100 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fb169-3d32-491c-b999-9133cc46ce09 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:07.841623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fb169-3d32-491c-b999-9133cc46ce09 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:07.842377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fb169-3d32-491c-b999-9133cc46ce09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:07.842683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fb169-3d32-491c-b999-9133cc46ce09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:07.844965 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fb169-3d32-491c-b999-9133cc46ce09 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:07.863120 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 207/406] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:07.864141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fb169-3d32-491c-b999-9133cc46ce09 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:07.904834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339104ba-549f-401e-bdef-5242216e4e83 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:07.924689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:07.947799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a6a0f-af98-4916-81c5-02ec8fb049d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:07.948571 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a6a0f-af98-4916-81c5-02ec8fb049d3 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:07.948966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a6a0f-af98-4916-81c5-02ec8fb049d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:07.949343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a6a0f-af98-4916-81c5-02ec8fb049d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:07.950139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a6a0f-af98-4916-81c5-02ec8fb049d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:07.950503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a6a0f-af98-4916-81c5-02ec8fb049d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:07.952702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a6a0f-af98-4916-81c5-02ec8fb049d3 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:07.962214 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 200/407] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:07.963201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a6a0f-af98-4916-81c5-02ec8fb049d3 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:08.029796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fb169-3d32-491c-b999-9133cc46ce09 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:08.076055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716183d5-1afc-4892-b25d-c3b791d41ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:08.077245 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716183d5-1afc-4892-b25d-c3b791d41ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:08.078001 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716183d5-1afc-4892-b25d-c3b791d41ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:08.078506 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716183d5-1afc-4892-b25d-c3b791d41ce1 None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:08.079872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716183d5-1afc-4892-b25d-c3b791d41ce1 None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:08.082108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716183d5-1afc-4892-b25d-c3b791d41ce1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:08.083803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 208/408] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:07 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:08.084604 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716183d5-1afc-4892-b25d-c3b791d41ce1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:08.090911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:08.098797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a6a0f-af98-4916-81c5-02ec8fb049d3 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:08.135523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7720f-1b04-4a14-985c-718d28b5a11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:08.136762 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7720f-1b04-4a14-985c-718d28b5a11a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:08.137419 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7720f-1b04-4a14-985c-718d28b5a11a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:08.137922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7720f-1b04-4a14-985c-718d28b5a11a None admin] PATH_INFO: `/v3/groups/9687a8d318304e8c8dba3eac7d1eba4b` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:08.139614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7720f-1b04-4a14-985c-718d28b5a11a None admin] RBAC: Authorizing `identity:get_group(group_id=9687a8d318304e8c8dba3eac7d1eba4b)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:08.141309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7720f-1b04-4a14-985c-718d28b5a11a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:08.143886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 201/409] 10.222.0.70 () {56 vars in 1180 bytes} [Wed Aug 24 16:28:07 2022] GET /identity/v3/groups/9687a8d318304e8c8dba3eac7d1eba4b => generated 229 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:08.145030 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7720f-1b04-4a14-985c-718d28b5a11a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:08.194304 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716183d5-1afc-4892-b25d-c3b791d41ce1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:08.197137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:08.235393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be5f9c-1073-497f-b6b8-4ed80fbf56cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:08.236017 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be5f9c-1073-497f-b6b8-4ed80fbf56cd None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:08.236292 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be5f9c-1073-497f-b6b8-4ed80fbf56cd None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:08.236594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be5f9c-1073-497f-b6b8-4ed80fbf56cd None admin] PATH_INFO: `/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:08.237874 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be5f9c-1073-497f-b6b8-4ed80fbf56cd None admin] RBAC: Authorizing `identity:get_group(group_id=f37e627622cc425dbf2e0b66f91d2ac2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:08.239365 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be5f9c-1073-497f-b6b8-4ed80fbf56cd None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:08.246214 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 209/410] 10.222.0.70 () {56 vars in 1180 bytes} [Wed Aug 24 16:28:08 2022] GET /identity/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:08.247144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be5f9c-1073-497f-b6b8-4ed80fbf56cd None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:08.259415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7720f-1b04-4a14-985c-718d28b5a11a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:08.298581 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ab56c-cf52-4694-98cd-6e0510a80b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:08.299658 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ab56c-cf52-4694-98cd-6e0510a80b24 None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:08.300140 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ab56c-cf52-4694-98cd-6e0510a80b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:08.300701 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ab56c-cf52-4694-98cd-6e0510a80b24 None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15/groups/f37e627622cc425dbf2e0b66f91d2ac2/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:08.302903 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ab56c-cf52-4694-98cd-6e0510a80b24 None admin] RBAC: Authorizing `identity:create_grant(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15, group_id=f37e627622cc425dbf2e0b66f91d2ac2, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:08.304936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ab56c-cf52-4694-98cd-6e0510a80b24 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:08.372023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be5f9c-1073-497f-b6b8-4ed80fbf56cd None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:08.412909 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b73f4a-4fa8-4960-995b-11718918ef02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:08.413559 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b73f4a-4fa8-4960-995b-11718918ef02 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:08.413828 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b73f4a-4fa8-4960-995b-11718918ef02 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:08.414101 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b73f4a-4fa8-4960-995b-11718918ef02 None admin] PATH_INFO: `/v3/projects/8f3b91fa60b4447a866a572209ebc37f` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:08.418819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b73f4a-4fa8-4960-995b-11718918ef02 None admin] RBAC: Authorizing `identity:get_project(project_id=8f3b91fa60b4447a866a572209ebc37f)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:08.418819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b73f4a-4fa8-4960-995b-11718918ef02 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:08.418819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 210/412] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:08 2022] GET /identity/v3/projects/8f3b91fa60b4447a866a572209ebc37f => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:08.419639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 202/412] 10.222.0.70 () {58 vars in 1442 bytes} [Wed Aug 24 16:28:08 2022] PUT /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15/groups/f37e627622cc425dbf2e0b66f91d2ac2/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:28:08.419743 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b73f4a-4fa8-4960-995b-11718918ef02 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:08.426064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ab56c-cf52-4694-98cd-6e0510a80b24 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:08.468874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:08.572941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ab56c-cf52-4694-98cd-6e0510a80b24 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:08.588820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:08.618771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cdbedf-9f4f-47db-a1ca-fea5abc6d33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:08.619819 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdbedf-9f4f-47db-a1ca-fea5abc6d33c None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:08.620283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdbedf-9f4f-47db-a1ca-fea5abc6d33c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:08.620990 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdbedf-9f4f-47db-a1ca-fea5abc6d33c None admin] PATH_INFO: `/v3/projects/8f3b91fa60b4447a866a572209ebc37f/groups/9687a8d318304e8c8dba3eac7d1eba4b/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:08.625486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cdbedf-9f4f-47db-a1ca-fea5abc6d33c None admin] RBAC: Authorizing `identity:create_grant(project_id=8f3b91fa60b4447a866a572209ebc37f, group_id=9687a8d318304e8c8dba3eac7d1eba4b, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:08.628943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cdbedf-9f4f-47db-a1ca-fea5abc6d33c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:08.703327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b73f4a-4fa8-4960-995b-11718918ef02 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:08.718494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 203/413] 10.222.0.70 () {58 vars in 1442 bytes} [Wed Aug 24 16:28:08 2022] PUT /identity/v3/projects/8f3b91fa60b4447a866a572209ebc37f/groups/9687a8d318304e8c8dba3eac7d1eba4b/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:28:08.760276 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461eb0ba-ae79-40dd-89af-5fcba968040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:08.761183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461eb0ba-ae79-40dd-89af-5fcba968040f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:08.761584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461eb0ba-ae79-40dd-89af-5fcba968040f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:08.761955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461eb0ba-ae79-40dd-89af-5fcba968040f None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:08.763365 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461eb0ba-ae79-40dd-89af-5fcba968040f None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:08.765549 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461eb0ba-ae79-40dd-89af-5fcba968040f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:08.767996 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 211/414] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:08 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:08.775274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cdbedf-9f4f-47db-a1ca-fea5abc6d33c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:08.893874 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30034b-906d-407b-bf26-f30901c6fe9d None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:08.894243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30034b-906d-407b-bf26-f30901c6fe9d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:08.894694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30034b-906d-407b-bf26-f30901c6fe9d None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:08.896573 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 212/415] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:08.899854 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:08.903137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619539b-d7c1-461e-ac36-80cbeb782fdc None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:08.903630 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619539b-d7c1-461e-ac36-80cbeb782fdc None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:08.904118 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619539b-d7c1-461e-ac36-80cbeb782fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:08.950711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9619539b-d7c1-461e-ac36-80cbeb782fdc None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:08.953634 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9619539b-d7c1-461e-ac36-80cbeb782fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:08.956765 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cdbedf-9f4f-47db-a1ca-fea5abc6d33c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:09.019840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bfb910-9620-4745-ba3b-de421da944b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:09.021020 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfb910-9620-4745-ba3b-de421da944b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:09.021458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfb910-9620-4745-ba3b-de421da944b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:09.021879 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfb910-9620-4745-ba3b-de421da944b0 None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15/groups/f37e627622cc425dbf2e0b66f91d2ac2/roles/003cfd128ecf4c238197991ad36f0c9a` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:09.025647 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bfb910-9620-4745-ba3b-de421da944b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15, group_id=f37e627622cc425dbf2e0b66f91d2ac2, role_id=003cfd128ecf4c238197991ad36f0c9a)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:09.027477 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bfb910-9620-4745-ba3b-de421da944b0 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:09.066888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 213/416] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:28:09.111230 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 204/417] 10.222.0.70 () {58 vars in 1442 bytes} [Wed Aug 24 16:28:08 2022] PUT /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15/groups/f37e627622cc425dbf2e0b66f91d2ac2/roles/003cfd128ecf4c238197991ad36f0c9a => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:28:09.457689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619539b-d7c1-461e-ac36-80cbeb782fdc None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:09.480125 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaa307-34d2-4f9c-9e8a-127692e532ea None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:09.480437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaa307-34d2-4f9c-9e8a-127692e532ea None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:09.480738 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaa307-34d2-4f9c-9e8a-127692e532ea None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:09.483738 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 205/418] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:09.489414 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b974113-d184-435a-a8bc-8d27316cc293 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:09.489719 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b974113-d184-435a-a8bc-8d27316cc293 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:09.489972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b974113-d184-435a-a8bc-8d27316cc293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:09.510082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:09.510637 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-1b974113-d184-435a-a8bc-8d27316cc293 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:09.512265 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1b974113-d184-435a-a8bc-8d27316cc293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:09.601079 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619539b-d7c1-461e-ac36-80cbeb782fdc None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:09.602120 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 206/419] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:28:09.646356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7fc5c-c76b-4086-addd-2e79b54f9a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:09.647562 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7fc5c-c76b-4086-addd-2e79b54f9a60 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:09.648195 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7fc5c-c76b-4086-addd-2e79b54f9a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:09.648748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7fc5c-c76b-4086-addd-2e79b54f9a60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:09.661661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7fc5c-c76b-4086-addd-2e79b54f9a60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f8b7fc5c-c76b-4086-addd-2e79b54f9a60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:09.667272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:09.668129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:09.668955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:09.668955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:09.668955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:09.668955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:09.668955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:09.668955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:09.668955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:09.668955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:09.668955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:09.668955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:09.668955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:28:09.668955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:09.673745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 214/420] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:09.677897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b974113-d184-435a-a8bc-8d27316cc293 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:09.757530 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:09.796777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b974113-d184-435a-a8bc-8d27316cc293 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:09.837046 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46de869-2bd7-4746-b3d7-7bf9535d7964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:09.838014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46de869-2bd7-4746-b3d7-7bf9535d7964 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:09.838550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46de869-2bd7-4746-b3d7-7bf9535d7964 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:09.839020 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46de869-2bd7-4746-b3d7-7bf9535d7964 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:09.839897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46de869-2bd7-4746-b3d7-7bf9535d7964 None admin] RBAC: Adding query filter params (name=member) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:09.840458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46de869-2bd7-4746-b3d7-7bf9535d7964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:09.842204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46de869-2bd7-4746-b3d7-7bf9535d7964 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:09.856634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 207/421] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:28:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:09.859227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7fc5c-c76b-4086-addd-2e79b54f9a60 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:09.965404 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46de869-2bd7-4746-b3d7-7bf9535d7964 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:10.012811 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:10.022781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7fc5c-c76b-4086-addd-2e79b54f9a60 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:10.114988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7167e3-fef0-4469-9be2-8d4906483654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:10.115621 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7167e3-fef0-4469-9be2-8d4906483654 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:10.115901 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7167e3-fef0-4469-9be2-8d4906483654 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:10.116172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7167e3-fef0-4469-9be2-8d4906483654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:10.167691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46de869-2bd7-4746-b3d7-7bf9535d7964 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:10.237188 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e4e8d-03d0-4945-a3f8-204c8490d2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:10.237856 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e4e8d-03d0-4945-a3f8-204c8490d2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:10.238146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e4e8d-03d0-4945-a3f8-204c8490d2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:10.238424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e4e8d-03d0-4945-a3f8-204c8490d2c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:10.242231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:10.249820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7167e3-fef0-4469-9be2-8d4906483654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:10.251419 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7167e3-fef0-4469-9be2-8d4906483654 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:10.260440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e4e8d-03d0-4945-a3f8-204c8490d2c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-f77e4e8d-03d0-4945-a3f8-204c8490d2c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:10.271936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:10.273525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:10.275006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:10.275006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:10.275006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:10.275006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:10.275006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:10.275006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:10.275006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:10.275006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:10.275006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:10.275006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:10.275006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:28:10.275006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:10.276136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 208/422] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:28:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:28:10.283195 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e4e8d-03d0-4945-a3f8-204c8490d2c7 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:10.452783 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:10.457892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 215/423] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:10.464743 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7167e3-fef0-4469-9be2-8d4906483654 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:10.475779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e4e8d-03d0-4945-a3f8-204c8490d2c7 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:10.540128 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f515802-1ef7-418c-b745-0e3de8a0ccd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:10.540702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f515802-1ef7-418c-b745-0e3de8a0ccd1 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:10.540987 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f515802-1ef7-418c-b745-0e3de8a0ccd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:10.541261 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f515802-1ef7-418c-b745-0e3de8a0ccd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:10.541950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f515802-1ef7-418c-b745-0e3de8a0ccd1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:10.542257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f515802-1ef7-418c-b745-0e3de8a0ccd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:10.543669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f515802-1ef7-418c-b745-0e3de8a0ccd1 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:10.566337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 209/424] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:10.571252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f515802-1ef7-418c-b745-0e3de8a0ccd1 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:10.689958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7167e3-fef0-4469-9be2-8d4906483654 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:10.704791 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:10.764019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fb26f-c4b1-4bf3-82fb-6d50365e417e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:10.765521 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fb26f-c4b1-4bf3-82fb-6d50365e417e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:10.765963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fb26f-c4b1-4bf3-82fb-6d50365e417e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:10.766688 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fb26f-c4b1-4bf3-82fb-6d50365e417e None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:10.768340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fb26f-c4b1-4bf3-82fb-6d50365e417e None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:10.770037 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fb26f-c4b1-4bf3-82fb-6d50365e417e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:10.773466 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 216/425] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:10 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:28:10.779291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fb26f-c4b1-4bf3-82fb-6d50365e417e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:10.793031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f515802-1ef7-418c-b745-0e3de8a0ccd1 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:10.862790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b51f41-349b-4853-bb3c-eaca6c1033f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:10.863717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b51f41-349b-4853-bb3c-eaca6c1033f2 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:10.864173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b51f41-349b-4853-bb3c-eaca6c1033f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:10.864617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b51f41-349b-4853-bb3c-eaca6c1033f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:10.913308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:10.983556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fb26f-c4b1-4bf3-82fb-6d50365e417e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:11.008768 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b51f41-349b-4853-bb3c-eaca6c1033f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:11.010683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b51f41-349b-4853-bb3c-eaca6c1033f2 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:11.046947 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555da5c-e950-48e1-b97f-f006db7cafbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:11.048166 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555da5c-e950-48e1-b97f-f006db7cafbb None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:11.048518 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555da5c-e950-48e1-b97f-f006db7cafbb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:11.048799 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555da5c-e950-48e1-b97f-f006db7cafbb None admin] PATH_INFO: `/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:11.050068 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555da5c-e950-48e1-b97f-f006db7cafbb None admin] RBAC: Authorizing `identity:get_group(group_id=f37e627622cc425dbf2e0b66f91d2ac2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:11.051863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555da5c-e950-48e1-b97f-f006db7cafbb None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:11.054787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 217/426] 10.222.0.70 () {56 vars in 1180 bytes} [Wed Aug 24 16:28:10 2022] GET /identity/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:11.056607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bff216-bb59-4834-aa33-1e6cc8c8f5c8 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:11.056897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bff216-bb59-4834-aa33-1e6cc8c8f5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:11.057175 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bff216-bb59-4834-aa33-1e6cc8c8f5c8 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:11.060660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 218/427] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:11.062236 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bff216-bb59-4834-aa33-1e6cc8c8f5c8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:11.094681 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:11.148292 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 210/428] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:11.153205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16742db8-9da1-4831-ae6c-1402477d099d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:11.153205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16742db8-9da1-4831-ae6c-1402477d099d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:11.153205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16742db8-9da1-4831-ae6c-1402477d099d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:11.171252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-16742db8-9da1-4831-ae6c-1402477d099d None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:11.172782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-16742db8-9da1-4831-ae6c-1402477d099d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:11.183751 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bff216-bb59-4834-aa33-1e6cc8c8f5c8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:11.235385 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4cc57-4b12-4e23-9813-08eae8fa9bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:11.236512 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4cc57-4b12-4e23-9813-08eae8fa9bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:11.237108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4cc57-4b12-4e23-9813-08eae8fa9bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:11.237592 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4cc57-4b12-4e23-9813-08eae8fa9bc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:11.238615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4cc57-4b12-4e23-9813-08eae8fa9bc0 None admin] RBAC: Adding query filter params (group.id=f37e627622cc425dbf2e0b66f91d2ac2, role.id=c7b826398b4e466491d4723321aaa5f1, scope.project.id=41be92ad8eba49128ba42ffcedab9caf) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:11.239167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4cc57-4b12-4e23-9813-08eae8fa9bc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:11.241001 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4cc57-4b12-4e23-9813-08eae8fa9bc0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:11.243442 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:11.249808 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 219/429] 10.222.0.70 () {56 vars in 1376 bytes} [Wed Aug 24 16:28:11 2022] GET /identity/v3/role_assignments?group.id=f37e627622cc425dbf2e0b66f91d2ac2&scope.project.id=41be92ad8eba49128ba42ffcedab9caf&role.id=c7b826398b4e466491d4723321aaa5f1 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:11.251240 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4cc57-4b12-4e23-9813-08eae8fa9bc0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:11.268050 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 211/430] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:11.378037 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4cc57-4b12-4e23-9813-08eae8fa9bc0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:11.408291 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170ff9e-4ef9-49de-a460-b310de165e08 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:11.408640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170ff9e-4ef9-49de-a460-b310de165e08 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:11.408899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170ff9e-4ef9-49de-a460-b310de165e08 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:11.410924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 212/431] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:11.417273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b61c9-7e38-4b16-b598-415ab5ff837a None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:11.417582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b61c9-7e38-4b16-b598-415ab5ff837a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:11.417835 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b61c9-7e38-4b16-b598-415ab5ff837a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:11.420293 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd61766-ae6d-4a92-8f2f-469626a19262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:11.421802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd61766-ae6d-4a92-8f2f-469626a19262 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:11.422347 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd61766-ae6d-4a92-8f2f-469626a19262 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:11.422826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd61766-ae6d-4a92-8f2f-469626a19262 None admin] PATH_INFO: `/v3/projects/41be92ad8eba49128ba42ffcedab9caf` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:11.424276 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd61766-ae6d-4a92-8f2f-469626a19262 None admin] RBAC: Authorizing `identity:get_project(project_id=41be92ad8eba49128ba42ffcedab9caf)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:11.426030 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd61766-ae6d-4a92-8f2f-469626a19262 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:11.428048 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 220/432] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:11 2022] GET /identity/v3/projects/41be92ad8eba49128ba42ffcedab9caf => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:11.434455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd61766-ae6d-4a92-8f2f-469626a19262 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:11.439558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-7b0b61c9-7e38-4b16-b598-415ab5ff837a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:11.440858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7b0b61c9-7e38-4b16-b598-415ab5ff837a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:11.518365 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:11.591405 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 213/433] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:28:11.592655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd61766-ae6d-4a92-8f2f-469626a19262 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:11.631064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798525bf-9aee-4cf3-8733-62c651f649bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:11.631628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798525bf-9aee-4cf3-8733-62c651f649bf None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:11.632015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798525bf-9aee-4cf3-8733-62c651f649bf None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:11.632368 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798525bf-9aee-4cf3-8733-62c651f649bf None admin] PATH_INFO: `/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:11.633578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798525bf-9aee-4cf3-8733-62c651f649bf None admin] RBAC: Authorizing `identity:get_group(group_id=f37e627622cc425dbf2e0b66f91d2ac2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:11.635069 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798525bf-9aee-4cf3-8733-62c651f649bf None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:11.637064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 221/434] 10.222.0.70 () {56 vars in 1180 bytes} [Wed Aug 24 16:28:11 2022] GET /identity/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:11.642844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b61c9-7e38-4b16-b598-415ab5ff837a None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:11.671335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:11.704812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798525bf-9aee-4cf3-8733-62c651f649bf None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:11.791673 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b61c9-7e38-4b16-b598-415ab5ff837a None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:11.847701 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798525bf-9aee-4cf3-8733-62c651f649bf None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:11.867434 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e5636-8903-483e-bdaf-59587fabe5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:11.868515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e5636-8903-483e-bdaf-59587fabe5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:11.869010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e5636-8903-483e-bdaf-59587fabe5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:11.869498 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e5636-8903-483e-bdaf-59587fabe5a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:11.870753 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e5636-8903-483e-bdaf-59587fabe5a2 None admin] RBAC: Adding query filter params (group.id=f37e627622cc425dbf2e0b66f91d2ac2, role.id=003cfd128ecf4c238197991ad36f0c9a, scope.project.id=41be92ad8eba49128ba42ffcedab9caf) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:11.871296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e5636-8903-483e-bdaf-59587fabe5a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:11.875671 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e5636-8903-483e-bdaf-59587fabe5a2 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:11.883877 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 214/435] 10.222.0.70 () {56 vars in 1376 bytes} [Wed Aug 24 16:28:11 2022] GET /identity/v3/role_assignments?group.id=f37e627622cc425dbf2e0b66f91d2ac2&scope.project.id=41be92ad8eba49128ba42ffcedab9caf&role.id=003cfd128ecf4c238197991ad36f0c9a => generated 611 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:11.899770 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8969f64-711c-4025-bfc9-825c5e244d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:11.900261 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e5636-8903-483e-bdaf-59587fabe5a2 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:11.900513 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8969f64-711c-4025-bfc9-825c5e244d65 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:11.900840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8969f64-711c-4025-bfc9-825c5e244d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:11.901016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8969f64-711c-4025-bfc9-825c5e244d65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:11.903699 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:11.921779 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8969f64-711c-4025-bfc9-825c5e244d65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-e8969f64-711c-4025-bfc9-825c5e244d65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:11.926941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:11.927937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:11.928791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:11.928791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:11.928791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:11.928791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:11.928791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:11.928791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:11.928791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:11.928791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:11.928791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:11.928791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:11.928791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:28:11.928791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:11.928791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 222/436] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:28:11.929951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e963ab-e85b-4940-bccd-2c57655c1fcd None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:11.930229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e963ab-e85b-4940-bccd-2c57655c1fcd None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:11.930507 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e963ab-e85b-4940-bccd-2c57655c1fcd None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:11.932144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 223/437] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:11.936617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e963ab-e85b-4940-bccd-2c57655c1fcd None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:12.033556 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:12.073671 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e5636-8903-483e-bdaf-59587fabe5a2 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:12.118155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793b9c8-3ee4-43fd-a2c2-a92899898dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:12.118804 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b9c8-3ee4-43fd-a2c2-a92899898dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:12.119088 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b9c8-3ee4-43fd-a2c2-a92899898dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:12.119458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b9c8-3ee4-43fd-a2c2-a92899898dc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:12.123440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e963ab-e85b-4940-bccd-2c57655c1fcd None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:12.131324 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793b9c8-3ee4-43fd-a2c2-a92899898dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-5793b9c8-3ee4-43fd-a2c2-a92899898dc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:12.136858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:12.138560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:12.139385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:12.139385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:12.139385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:12.139385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:12.139385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:12.139385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:12.139385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:12.139385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:12.139385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:12.139385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:12.139385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:12.139385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:12.139385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 215/438] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:12.139385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d12330-8102-4565-814b-8f6fa3b22018 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:12.140096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d12330-8102-4565-814b-8f6fa3b22018 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:12.140096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d12330-8102-4565-814b-8f6fa3b22018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:12.160262 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-06d12330-8102-4565-814b-8f6fa3b22018 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:12.161419 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-06d12330-8102-4565-814b-8f6fa3b22018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:12.183132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba90ef-3f40-4cd0-98af-623ae77937a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:12.184186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba90ef-3f40-4cd0-98af-623ae77937a8 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:12.184500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba90ef-3f40-4cd0-98af-623ae77937a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:12.184782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba90ef-3f40-4cd0-98af-623ae77937a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:12.185470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ba90ef-3f40-4cd0-98af-623ae77937a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:12.185910 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ba90ef-3f40-4cd0-98af-623ae77937a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:12.187586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ba90ef-3f40-4cd0-98af-623ae77937a8 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:12.203283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 224/439] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:28:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:28:12.203283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba90ef-3f40-4cd0-98af-623ae77937a8 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:12.266972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 216/440] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:12.267855 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d12330-8102-4565-814b-8f6fa3b22018 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:12.278109 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:12.334833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba90ef-3f40-4cd0-98af-623ae77937a8 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:12.379338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269c0a8-404f-45e2-94eb-56eb7b5698c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:12.379969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269c0a8-404f-45e2-94eb-56eb7b5698c6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:12.380252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269c0a8-404f-45e2-94eb-56eb7b5698c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:12.380571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269c0a8-404f-45e2-94eb-56eb7b5698c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:12.381267 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269c0a8-404f-45e2-94eb-56eb7b5698c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:12.381577 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269c0a8-404f-45e2-94eb-56eb7b5698c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:12.383052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269c0a8-404f-45e2-94eb-56eb7b5698c6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:12.384210 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:12.399351 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 225/441] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:12.399351 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269c0a8-404f-45e2-94eb-56eb7b5698c6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:12.424062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d12330-8102-4565-814b-8f6fa3b22018 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:12.469264 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865e37e-bb07-4d26-937d-5873944bc60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:12.470116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865e37e-bb07-4d26-937d-5873944bc60c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:12.470576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865e37e-bb07-4d26-937d-5873944bc60c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:12.470951 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865e37e-bb07-4d26-937d-5873944bc60c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:12.575101 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865e37e-bb07-4d26-937d-5873944bc60c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:12.577129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865e37e-bb07-4d26-937d-5873944bc60c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:12.577715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269c0a8-404f-45e2-94eb-56eb7b5698c6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:12.602858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:12.633240 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e34b5-2b00-4793-a8e5-c175aeeb9c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:12.634265 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e34b5-2b00-4793-a8e5-c175aeeb9c7b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:12.634759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e34b5-2b00-4793-a8e5-c175aeeb9c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:12.635243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e34b5-2b00-4793-a8e5-c175aeeb9c7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:12.662436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:12.698854 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 217/442] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:12.698854 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865e37e-bb07-4d26-937d-5873944bc60c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:12.740484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e34b5-2b00-4793-a8e5-c175aeeb9c7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:12.742653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e34b5-2b00-4793-a8e5-c175aeeb9c7b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:12.836213 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865e37e-bb07-4d26-937d-5873944bc60c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:12.854907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 226/443] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:12.855783 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e34b5-2b00-4793-a8e5-c175aeeb9c7b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:12.877899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26f34d-aaad-423d-b79a-e540d5d1deec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:12.878629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26f34d-aaad-423d-b79a-e540d5d1deec None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:12.878820 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26f34d-aaad-423d-b79a-e540d5d1deec None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:12.879091 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26f34d-aaad-423d-b79a-e540d5d1deec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:12.882575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:12.897722 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26f34d-aaad-423d-b79a-e540d5d1deec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-1b26f34d-aaad-423d-b79a-e540d5d1deec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:12.907709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:12.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:12.910692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:12.910692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:12.910692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:12.910692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:12.910692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:12.910692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:12.910692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:12.910692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:12.910692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:12.910692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:12.910692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:28:12.910692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:12.910692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 218/444] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:28:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:12.911771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26f34d-aaad-423d-b79a-e540d5d1deec None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:12.965562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:12.976981 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e34b5-2b00-4793-a8e5-c175aeeb9c7b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:13.020981 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f6be0-123a-4bd9-8595-cc931eb6195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:13.021631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f6be0-123a-4bd9-8595-cc931eb6195a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:13.021911 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f6be0-123a-4bd9-8595-cc931eb6195a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:13.022200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f6be0-123a-4bd9-8595-cc931eb6195a None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:13.023395 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f6be0-123a-4bd9-8595-cc931eb6195a None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:13.024890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f6be0-123a-4bd9-8595-cc931eb6195a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:13.026805 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 227/445] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:12 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:13.027452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f6be0-123a-4bd9-8595-cc931eb6195a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:13.061405 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26f34d-aaad-423d-b79a-e540d5d1deec None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:13.130480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4a829-933a-4778-956f-3ddcea98bc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:13.131389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a829-933a-4778-956f-3ddcea98bc7b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:13.131845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a829-933a-4778-956f-3ddcea98bc7b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:13.132257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a829-933a-4778-956f-3ddcea98bc7b None admin] PATH_INFO: `/v3/projects/8f3b91fa60b4447a866a572209ebc37f` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:13.134130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4a829-933a-4778-956f-3ddcea98bc7b None admin] RBAC: Authorizing `identity:get_project(project_id=8f3b91fa60b4447a866a572209ebc37f)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:13.135856 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4a829-933a-4778-956f-3ddcea98bc7b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:13.137986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 219/446] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:12 2022] GET /identity/v3/projects/8f3b91fa60b4447a866a572209ebc37f => generated 344 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:13.139944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4a829-933a-4778-956f-3ddcea98bc7b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:13.142397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:13.161892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f6be0-123a-4bd9-8595-cc931eb6195a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:13.211932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9b1e7-8be5-4233-bea8-f3c114a3d590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:13.212587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9b1e7-8be5-4233-bea8-f3c114a3d590 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:13.212872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9b1e7-8be5-4233-bea8-f3c114a3d590 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:13.213144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9b1e7-8be5-4233-bea8-f3c114a3d590 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:13.213822 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9b1e7-8be5-4233-bea8-f3c114a3d590 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:13.214126 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9b1e7-8be5-4233-bea8-f3c114a3d590 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:13.215522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9b1e7-8be5-4233-bea8-f3c114a3d590 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:13.226459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 228/447] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:13.227326 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9b1e7-8be5-4233-bea8-f3c114a3d590 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:13.279239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:13.286038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4a829-933a-4778-956f-3ddcea98bc7b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:13.345458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9b1e7-8be5-4233-bea8-f3c114a3d590 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:13.350770 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1430d-351c-4157-901b-fe571288e428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:13.351869 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1430d-351c-4157-901b-fe571288e428 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:13.352369 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1430d-351c-4157-901b-fe571288e428 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:13.352875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1430d-351c-4157-901b-fe571288e428 None admin] PATH_INFO: `/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:13.355656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1430d-351c-4157-901b-fe571288e428 None admin] RBAC: Authorizing `identity:get_group(group_id=f37e627622cc425dbf2e0b66f91d2ac2)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:13.359942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1430d-351c-4157-901b-fe571288e428 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:13.363111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 220/448] 10.222.0.70 () {56 vars in 1180 bytes} [Wed Aug 24 16:28:13 2022] GET /identity/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:13.364668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1430d-351c-4157-901b-fe571288e428 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:13.384218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7d540-b5d5-48ff-abb8-61037f54a2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:13.384904 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7d540-b5d5-48ff-abb8-61037f54a2ad None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:13.385195 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7d540-b5d5-48ff-abb8-61037f54a2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:13.385471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7d540-b5d5-48ff-abb8-61037f54a2ad None admin] PATH_INFO: `/v3/groups/9687a8d318304e8c8dba3eac7d1eba4b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:13.386709 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7d540-b5d5-48ff-abb8-61037f54a2ad None admin] RBAC: Authorizing `identity:get_group(group_id=9687a8d318304e8c8dba3eac7d1eba4b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:13.388167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7d540-b5d5-48ff-abb8-61037f54a2ad None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:13.389835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 229/449] 10.222.0.70 () {56 vars in 1180 bytes} [Wed Aug 24 16:28:13 2022] GET /identity/v3/groups/9687a8d318304e8c8dba3eac7d1eba4b => generated 229 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:13.390639 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7d540-b5d5-48ff-abb8-61037f54a2ad None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:13.412206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:13.501841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1430d-351c-4157-901b-fe571288e428 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:13.511566 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7d540-b5d5-48ff-abb8-61037f54a2ad None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:13.546971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b6b42-a76d-44ed-94b5-340485f30bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:13.547582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b6b42-a76d-44ed-94b5-340485f30bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:13.547856 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b6b42-a76d-44ed-94b5-340485f30bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:13.548118 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b6b42-a76d-44ed-94b5-340485f30bf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:13.552577 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e6e15-a9fd-4e49-9080-2b2d58823625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:13.553549 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e6e15-a9fd-4e49-9080-2b2d58823625 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:13.553938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e6e15-a9fd-4e49-9080-2b2d58823625 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:13.554335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e6e15-a9fd-4e49-9080-2b2d58823625 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:13.555358 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e6e15-a9fd-4e49-9080-2b2d58823625 None admin] RBAC: Adding query filter params (group.id=f37e627622cc425dbf2e0b66f91d2ac2, role.id=c7b826398b4e466491d4723321aaa5f1, scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:13.555679 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e6e15-a9fd-4e49-9080-2b2d58823625 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:13.557999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e6e15-a9fd-4e49-9080-2b2d58823625 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:13.567604 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 230/450] 10.222.0.70 () {56 vars in 1376 bytes} [Wed Aug 24 16:28:13 2022] GET /identity/v3/role_assignments?group.id=f37e627622cc425dbf2e0b66f91d2ac2&scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15&role.id=c7b826398b4e466491d4723321aaa5f1 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:13.569036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e6e15-a9fd-4e49-9080-2b2d58823625 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:13.604988 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:13.647914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b6b42-a76d-44ed-94b5-340485f30bf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:13.649922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b6b42-a76d-44ed-94b5-340485f30bf1 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:13.677231 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:13.712819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e6e15-a9fd-4e49-9080-2b2d58823625 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:13.770569 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae718e1-d5cf-4b4a-9382-a078a2b7113b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:13.771383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae718e1-d5cf-4b4a-9382-a078a2b7113b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:13.771812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae718e1-d5cf-4b4a-9382-a078a2b7113b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:13.772139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae718e1-d5cf-4b4a-9382-a078a2b7113b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:13.773076 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae718e1-d5cf-4b4a-9382-a078a2b7113b None admin] RBAC: Adding query filter params (group.id=9687a8d318304e8c8dba3eac7d1eba4b, role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=8f3b91fa60b4447a866a572209ebc37f) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:13.773413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae718e1-d5cf-4b4a-9382-a078a2b7113b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:13.774892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae718e1-d5cf-4b4a-9382-a078a2b7113b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:13.788622 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 231/451] 10.222.0.70 () {56 vars in 1376 bytes} [Wed Aug 24 16:28:13 2022] GET /identity/v3/role_assignments?group.id=9687a8d318304e8c8dba3eac7d1eba4b&scope.project.id=8f3b91fa60b4447a866a572209ebc37f&role.id=e7f48343fc1542979ab4459865109e30 => generated 611 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:13.800210 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 221/452] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:13.807296 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae718e1-d5cf-4b4a-9382-a078a2b7113b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:13.956536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae718e1-d5cf-4b4a-9382-a078a2b7113b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:14.000309 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e851d-d464-421a-9ebd-bb07b85192d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:14.001003 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e851d-d464-421a-9ebd-bb07b85192d8 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:14.001288 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e851d-d464-421a-9ebd-bb07b85192d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:14.001564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e851d-d464-421a-9ebd-bb07b85192d8 None admin] PATH_INFO: `/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:14.002842 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e851d-d464-421a-9ebd-bb07b85192d8 None admin] RBAC: Authorizing `identity:get_project(project_id=9dc6a9fb9f4d4531b1a8725c2f79bd15)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:14.004481 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e851d-d464-421a-9ebd-bb07b85192d8 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:14.006224 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 232/453] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:13 2022] GET /identity/v3/projects/9dc6a9fb9f4d4531b1a8725c2f79bd15 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:14.015404 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b6b42-a76d-44ed-94b5-340485f30bf1 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:14.090668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:14.127518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b6b42-a76d-44ed-94b5-340485f30bf1 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:14.178614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f27f73-21d5-4da0-a3cb-43bc047a5081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:14.179246 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f27f73-21d5-4da0-a3cb-43bc047a5081 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:14.179529 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f27f73-21d5-4da0-a3cb-43bc047a5081 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:14.179804 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f27f73-21d5-4da0-a3cb-43bc047a5081 None admin] PATH_INFO: `/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:14.180960 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f27f73-21d5-4da0-a3cb-43bc047a5081 None admin] RBAC: Authorizing `identity:get_group(group_id=f37e627622cc425dbf2e0b66f91d2ac2)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:14.182465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f27f73-21d5-4da0-a3cb-43bc047a5081 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:14.184088 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 222/454] 10.222.0.70 () {56 vars in 1180 bytes} [Wed Aug 24 16:28:14 2022] GET /identity/v3/groups/f37e627622cc425dbf2e0b66f91d2ac2 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:14.190747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e851d-d464-421a-9ebd-bb07b85192d8 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:14.206648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:14.303778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e851d-d464-421a-9ebd-bb07b85192d8 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:14.340630 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5393e-e8a0-4e80-895f-d1a216ab8345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:14.341286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5393e-e8a0-4e80-895f-d1a216ab8345 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:14.341515 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5393e-e8a0-4e80-895f-d1a216ab8345 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:14.341791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5393e-e8a0-4e80-895f-d1a216ab8345 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:14.343073 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5393e-e8a0-4e80-895f-d1a216ab8345 None admin] RBAC: Adding query filter params (group.id=f37e627622cc425dbf2e0b66f91d2ac2, role.id=003cfd128ecf4c238197991ad36f0c9a, scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:14.343372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5393e-e8a0-4e80-895f-d1a216ab8345 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:14.344915 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5393e-e8a0-4e80-895f-d1a216ab8345 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:14.351473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 233/455] 10.222.0.70 () {56 vars in 1376 bytes} [Wed Aug 24 16:28:14 2022] GET /identity/v3/role_assignments?group.id=f37e627622cc425dbf2e0b66f91d2ac2&scope.project.id=9dc6a9fb9f4d4531b1a8725c2f79bd15&role.id=003cfd128ecf4c238197991ad36f0c9a => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:28:17.492346 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba8e92-3bce-458e-8f0a-d2812fc442c5 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:17.493425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba8e92-3bce-458e-8f0a-d2812fc442c5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:17.493959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba8e92-3bce-458e-8f0a-d2812fc442c5 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:17.495549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 223/456] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:17.501835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b5856-04ba-4746-923e-edc8795d6cea None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:17.502331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b5856-04ba-4746-923e-edc8795d6cea None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:17.502432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b5856-04ba-4746-923e-edc8795d6cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:17.525934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5e1b5856-04ba-4746-923e-edc8795d6cea None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:17.527220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5e1b5856-04ba-4746-923e-edc8795d6cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:17.635240 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 234/457] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:17.798594 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d702217-ac0a-4789-a489-c267e18220f3 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:17.799240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d702217-ac0a-4789-a489-c267e18220f3 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:17.799740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d702217-ac0a-4789-a489-c267e18220f3 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:17.801934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 224/458] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:17.807181 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab3441-2ea4-4647-b1f0-12664aa401af None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:17.807478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab3441-2ea4-4647-b1f0-12664aa401af None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:17.807743 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab3441-2ea4-4647-b1f0-12664aa401af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:17.827888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:17.834829 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7bab3441-2ea4-4647-b1f0-12664aa401af None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:17.836047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7bab3441-2ea4-4647-b1f0-12664aa401af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:17.941991 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 235/459] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:17.984193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70527f6-1c4a-4fa6-9877-0ec79046d6fc None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:17.984193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70527f6-1c4a-4fa6-9877-0ec79046d6fc None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:17.984193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70527f6-1c4a-4fa6-9877-0ec79046d6fc None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:17.984193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 225/460] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:17.990029 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a31362-f64e-4ecc-a1b7-9a4e9e35a81a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:17.990306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a31362-f64e-4ecc-a1b7-9a4e9e35a81a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:17.990583 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a31362-f64e-4ecc-a1b7-9a4e9e35a81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:18.017980 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b4a31362-f64e-4ecc-a1b7-9a4e9e35a81a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:18.019196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b4a31362-f64e-4ecc-a1b7-9a4e9e35a81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:18.117733 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 236/461] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:28:18.126097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70527f6-1c4a-4fa6-9877-0ec79046d6fc None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:18.223719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a31362-f64e-4ecc-a1b7-9a4e9e35a81a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:18.283321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70527f6-1c4a-4fa6-9877-0ec79046d6fc None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:18.330857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940ab9e-7784-4668-b2e1-7851e65b21b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:18.332814 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940ab9e-7784-4668-b2e1-7851e65b21b0 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:18.333342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940ab9e-7784-4668-b2e1-7851e65b21b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:18.334080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940ab9e-7784-4668-b2e1-7851e65b21b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:18.349058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940ab9e-7784-4668-b2e1-7851e65b21b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:18.351811 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940ab9e-7784-4668-b2e1-7851e65b21b0 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:18.352719 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:18.356298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-9940ab9e-7784-4668-b2e1-7851e65b21b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:18.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:18.373902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:18.374820 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:18.374820 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:18.374820 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:18.377884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 226/462] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:18.380169 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09135155-cfdc-4a51-a2c7-1bb8ebe60611 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:18.380717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09135155-cfdc-4a51-a2c7-1bb8ebe60611 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:18.381187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09135155-cfdc-4a51-a2c7-1bb8ebe60611 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:18.383812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 227/463] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:18.384916 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09135155-cfdc-4a51-a2c7-1bb8ebe60611 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:18.437267 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a31362-f64e-4ecc-a1b7-9a4e9e35a81a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:18.509226 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca92790-de69-4c34-951b-08933d5c2897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:18.510324 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca92790-de69-4c34-951b-08933d5c2897 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:18.510829 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca92790-de69-4c34-951b-08933d5c2897 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:18.511330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca92790-de69-4c34-951b-08933d5c2897 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:18.526953 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09135155-cfdc-4a51-a2c7-1bb8ebe60611 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:18.531574 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca92790-de69-4c34-951b-08933d5c2897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:18.534379 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca92790-de69-4c34-951b-08933d5c2897 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-3ca92790-de69-4c34-951b-08933d5c2897 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:18.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:18.565213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:18.566042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:18.566042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:18.566042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:18.571266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 237/464] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:18.571266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd115b-7799-4467-be5a-6cec49b6398e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:18.571266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd115b-7799-4467-be5a-6cec49b6398e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:18.571266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd115b-7799-4467-be5a-6cec49b6398e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:18.589482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a10d1-8ef7-4423-9311-c50ccc2e93db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:18.590535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a10d1-8ef7-4423-9311-c50ccc2e93db None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:18.591026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a10d1-8ef7-4423-9311-c50ccc2e93db None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:18.591527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a10d1-8ef7-4423-9311-c50ccc2e93db None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:18.593055 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a10d1-8ef7-4423-9311-c50ccc2e93db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:18.593195 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a10d1-8ef7-4423-9311-c50ccc2e93db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:18.595463 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a10d1-8ef7-4423-9311-c50ccc2e93db None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:18.599434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3bfd115b-7799-4467-be5a-6cec49b6398e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:18.600694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3bfd115b-7799-4467-be5a-6cec49b6398e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:18.620842 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 228/465] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:18.622997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a10d1-8ef7-4423-9311-c50ccc2e93db None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:18.709644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:18.721925 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 238/466] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:18.721925 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd115b-7799-4467-be5a-6cec49b6398e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:18.807638 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:18.808511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a10d1-8ef7-4423-9311-c50ccc2e93db None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:18.849986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3e385-85db-467a-9006-5a18294e7f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:18.851156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3e385-85db-467a-9006-5a18294e7f73 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:18.851156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3e385-85db-467a-9006-5a18294e7f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:18.851156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3e385-85db-467a-9006-5a18294e7f73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:18.865307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd3e385-85db-467a-9006-5a18294e7f73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:18.866817 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd3e385-85db-467a-9006-5a18294e7f73 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:18.873484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd115b-7799-4467-be5a-6cec49b6398e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-3dd3e385-85db-467a-9006-5a18294e7f73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:18.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:18.893311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:18.894159 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:18.894159 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:18.894159 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:18.894876 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 229/467] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:18.895915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3e385-85db-467a-9006-5a18294e7f73 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:18.924048 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679d8eb-2466-4d57-bcd5-d46a066994f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:18.924756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679d8eb-2466-4d57-bcd5-d46a066994f0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:18.925168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679d8eb-2466-4d57-bcd5-d46a066994f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:18.925340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679d8eb-2466-4d57-bcd5-d46a066994f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:18.925965 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679d8eb-2466-4d57-bcd5-d46a066994f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:18.926274 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679d8eb-2466-4d57-bcd5-d46a066994f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:18.927700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679d8eb-2466-4d57-bcd5-d46a066994f0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:18.942129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 239/468] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:18.943740 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c293b2-a97d-4fcc-bfb5-50daa905d7d5 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:18.944032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c293b2-a97d-4fcc-bfb5-50daa905d7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:18.944317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c293b2-a97d-4fcc-bfb5-50daa905d7d5 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:18.947204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 240/469] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:18.947204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c293b2-a97d-4fcc-bfb5-50daa905d7d5 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:19.045524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:19.050573 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3e385-85db-467a-9006-5a18294e7f73 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:19.077245 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c293b2-a97d-4fcc-bfb5-50daa905d7d5 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:19.090246 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:19.092926 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05aeef-c62d-4b3c-97ee-ff3e5c933e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:19.093739 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05aeef-c62d-4b3c-97ee-ff3e5c933e56 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:19.094083 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05aeef-c62d-4b3c-97ee-ff3e5c933e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:19.094684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05aeef-c62d-4b3c-97ee-ff3e5c933e56 None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:19.095336 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05aeef-c62d-4b3c-97ee-ff3e5c933e56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:19.096980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05aeef-c62d-4b3c-97ee-ff3e5c933e56 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:19.120032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f0921-484a-4177-bf11-4c6a1c6be120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:19.120748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f0921-484a-4177-bf11-4c6a1c6be120 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:19.121143 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f0921-484a-4177-bf11-4c6a1c6be120 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:19.121421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f0921-484a-4177-bf11-4c6a1c6be120 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:19.122078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f0921-484a-4177-bf11-4c6a1c6be120 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:19.122389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f0921-484a-4177-bf11-4c6a1c6be120 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:19.123908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f0921-484a-4177-bf11-4c6a1c6be120 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:19.136712 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 241/470] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:19.137554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f0921-484a-4177-bf11-4c6a1c6be120 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:19.177930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 230/471] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:28:18 2022] POST /identity/v3/users => generated 248 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:28:19.179137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b4da6-9c1e-4416-a56f-29ff91849d14 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:19.179437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b4da6-9c1e-4416-a56f-29ff91849d14 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:19.179720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b4da6-9c1e-4416-a56f-29ff91849d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:19.205667 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-b08b4da6-9c1e-4416-a56f-29ff91849d14 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:19.207588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b08b4da6-9c1e-4416-a56f-29ff91849d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:19.284008 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f0921-484a-4177-bf11-4c6a1c6be120 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:19.329148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 231/472] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:19.329148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b4da6-9c1e-4416-a56f-29ff91849d14 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:19.353899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cfba7-e606-4fdb-a7bc-d9ca67585505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:19.355211 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cfba7-e606-4fdb-a7bc-d9ca67585505 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:19.355787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cfba7-e606-4fdb-a7bc-d9ca67585505 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:19.356532 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cfba7-e606-4fdb-a7bc-d9ca67585505 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:19.357885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cfba7-e606-4fdb-a7bc-d9ca67585505 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:19.360946 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cfba7-e606-4fdb-a7bc-d9ca67585505 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:19.472229 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:19.484819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 242/473] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:28:19 2022] POST /identity/v3/users => generated 248 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:28:19.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cfba7-e606-4fdb-a7bc-d9ca67585505 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:19.504831 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b4da6-9c1e-4416-a56f-29ff91849d14 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:19.543602 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:19.596455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87666797-1c4c-4492-982f-700e5bed8f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:19.596455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87666797-1c4c-4492-982f-700e5bed8f13 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:19.596455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87666797-1c4c-4492-982f-700e5bed8f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:19.596455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87666797-1c4c-4492-982f-700e5bed8f13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:19.618635 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87666797-1c4c-4492-982f-700e5bed8f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:19.621260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87666797-1c4c-4492-982f-700e5bed8f13 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-87666797-1c4c-4492-982f-700e5bed8f13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:19.672284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:19.674053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:19.676426 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:19.676426 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:19.676426 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:19.676426 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 232/474] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:19.686872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87666797-1c4c-4492-982f-700e5bed8f13 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:19.750727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cfba7-e606-4fdb-a7bc-d9ca67585505 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:19.801750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451ebbe-345c-45d0-9cb6-4210c425badf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:19.802386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451ebbe-345c-45d0-9cb6-4210c425badf None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:19.802766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451ebbe-345c-45d0-9cb6-4210c425badf None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:19.803065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451ebbe-345c-45d0-9cb6-4210c425badf None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:19.803891 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451ebbe-345c-45d0-9cb6-4210c425badf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:19.805452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451ebbe-345c-45d0-9cb6-4210c425badf None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:19.825078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87666797-1c4c-4492-982f-700e5bed8f13 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:19.869375 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483937b5-10f6-4235-b692-0c28827b83d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:19.870116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483937b5-10f6-4235-b692-0c28827b83d5 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:19.870403 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483937b5-10f6-4235-b692-0c28827b83d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:19.870726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483937b5-10f6-4235-b692-0c28827b83d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:19.871441 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483937b5-10f6-4235-b692-0c28827b83d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:19.871752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483937b5-10f6-4235-b692-0c28827b83d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:19.873818 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483937b5-10f6-4235-b692-0c28827b83d5 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:19.876819 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:19.896462 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 233/475] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:19.896462 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483937b5-10f6-4235-b692-0c28827b83d5 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:19.944159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 243/476] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:28:19 2022] POST /identity/v3/users => generated 246 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:28:19.949757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451ebbe-345c-45d0-9cb6-4210c425badf None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:20.110677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:20.119675 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483937b5-10f6-4235-b692-0c28827b83d5 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:20.134355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451ebbe-345c-45d0-9cb6-4210c425badf None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:20.173691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc6f0a-8998-45ea-a745-ad240d64d0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:20.175491 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc6f0a-8998-45ea-a745-ad240d64d0c2 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:20.175491 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc6f0a-8998-45ea-a745-ad240d64d0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:20.175491 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc6f0a-8998-45ea-a745-ad240d64d0c2 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:20.176214 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc6f0a-8998-45ea-a745-ad240d64d0c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:20.178174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc6f0a-8998-45ea-a745-ad240d64d0c2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:20.180886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e238bbb-9587-4d05-ba17-e6d43dfd826a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:20.181747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e238bbb-9587-4d05-ba17-e6d43dfd826a None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:20.182039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e238bbb-9587-4d05-ba17-e6d43dfd826a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:20.182321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e238bbb-9587-4d05-ba17-e6d43dfd826a None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:20.183001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e238bbb-9587-4d05-ba17-e6d43dfd826a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:20.184425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e238bbb-9587-4d05-ba17-e6d43dfd826a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:20.234925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.sql.core [None req-1e238bbb-9587-4d05-ba17-e6d43dfd826a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 16:28:20.234925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 16:28:20.234925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [parameters: {'id': 'ed076412cd6f4479960df5efddda7005', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 16:28:20.234925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81844) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 16:28:20.254431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-1e238bbb-9587-4d05-ba17-e6d43dfd826a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 16:28:20.254431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:20.254431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:28:20.254431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:28:20.254431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:28:20.254431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:28:20.254431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:28:20.254431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:28:20.254431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:28:20.254431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:28:20.254431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:28:20.254431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:28:20.254431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:28:20.254431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application result.read() Aug 24 16:28:20.254431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:28:20.255943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application yield resource Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise self.value Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application session.commit() Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 16:28:20.257295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self.session.flush() Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application compat.raise_( Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 16:28:20.258924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:28:20.260382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application result.read() Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:28:20.261642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [parameters: {'id': 'ed076412cd6f4479960df5efddda7005', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:20.262754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 16:28:20.263795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:20.265403 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 234/477] 10.222.0.70 () {60 vars in 1131 bytes} [Wed Aug 24 16:28:19 2022] POST /identity/v3/roles => generated 144 bytes in 365 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:20.267326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e238bbb-9587-4d05-ba17-e6d43dfd826a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:20.287767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 244/478] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:28:19 2022] POST /identity/v3/users => generated 249 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:28:20.349777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:20.403597 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e238bbb-9587-4d05-ba17-e6d43dfd826a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:20.446637 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422edd0a-3dfa-41cf-90b6-bfd565755202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:20.447617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422edd0a-3dfa-41cf-90b6-bfd565755202 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:20.448118 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422edd0a-3dfa-41cf-90b6-bfd565755202 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:20.448587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422edd0a-3dfa-41cf-90b6-bfd565755202 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:20.462688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422edd0a-3dfa-41cf-90b6-bfd565755202 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-422edd0a-3dfa-41cf-90b6-bfd565755202 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:20.471388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:20.473412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:20.475184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:20.475184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:20.475184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:20.475184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:20.475184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:20.475184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:20.475184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:20.475184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:20.475184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:20.475184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:20.475184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:28:20.475184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:20.483629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 235/479] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:28:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:20.489707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc6f0a-8998-45ea-a745-ad240d64d0c2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:20.689936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc6f0a-8998-45ea-a745-ad240d64d0c2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:20.734239 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b2283-4fdc-4422-8079-9f128d82d214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:20.735130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b2283-4fdc-4422-8079-9f128d82d214 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:20.735559 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b2283-4fdc-4422-8079-9f128d82d214 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:20.736000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b2283-4fdc-4422-8079-9f128d82d214 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:20.737175 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b2283-4fdc-4422-8079-9f128d82d214 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:20.737648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b2283-4fdc-4422-8079-9f128d82d214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:20.739210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b2283-4fdc-4422-8079-9f128d82d214 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:20.743193 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:20.756633 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 245/480] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:21.839342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66f060-08e3-4865-84be-6ea4a9e2e11c None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:21.839858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66f060-08e3-4865-84be-6ea4a9e2e11c None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:21.839858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66f060-08e3-4865-84be-6ea4a9e2e11c None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:21.842849 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 236/481] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:21.849047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7f525-14cc-475b-8497-fbcf8dd081fb None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:21.849645 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7f525-14cc-475b-8497-fbcf8dd081fb None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:21.849645 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7f525-14cc-475b-8497-fbcf8dd081fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:21.864361 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56236405-02af-421f-9e91-d612eea5fd28 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:21.865018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56236405-02af-421f-9e91-d612eea5fd28 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:21.865519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56236405-02af-421f-9e91-d612eea5fd28 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:21.867548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 237/482] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:21.873752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12291eca-67e5-407d-8c09-51301a829a33 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:21.874032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12291eca-67e5-407d-8c09-51301a829a33 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:21.874308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12291eca-67e5-407d-8c09-51301a829a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:21.878997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f3b7f525-14cc-475b-8497-fbcf8dd081fb None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:21.880552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f3b7f525-14cc-475b-8497-fbcf8dd081fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:21.903311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-12291eca-67e5-407d-8c09-51301a829a33 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:21.905097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-12291eca-67e5-407d-8c09-51301a829a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:22.028150 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 246/483] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:22.054963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 238/484] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:22.434393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7f525-14cc-475b-8497-fbcf8dd081fb None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:22.535549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12291eca-67e5-407d-8c09-51301a829a33 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:22.667521 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7f525-14cc-475b-8497-fbcf8dd081fb None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:22.688572 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:22.709629 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:22.718371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548c87b-4201-4cb8-b917-b546f1134478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:22.719278 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548c87b-4201-4cb8-b917-b546f1134478 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:22.719623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548c87b-4201-4cb8-b917-b546f1134478 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:22.719840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548c87b-4201-4cb8-b917-b546f1134478 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:22.734482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4548c87b-4201-4cb8-b917-b546f1134478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-4548c87b-4201-4cb8-b917-b546f1134478 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:22.741625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:22.742898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:22.743754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:22.743754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:22.743754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:22.743754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:22.743754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:22.743754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:22.743754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:22.743754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:22.743754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:22.743754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:22.743754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:22.743754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:22.746076 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 247/485] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:22.747473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee2e7e-524d-4ad6-a834-f31c40b35eba None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:22.748307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee2e7e-524d-4ad6-a834-f31c40b35eba None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:22.748611 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee2e7e-524d-4ad6-a834-f31c40b35eba None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:22.750342 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 248/486] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:22.760433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cc6c6-9877-492d-9fd3-ef92687560cc None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:22.760433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cc6c6-9877-492d-9fd3-ef92687560cc None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:22.760433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cc6c6-9877-492d-9fd3-ef92687560cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:22.766070 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12291eca-67e5-407d-8c09-51301a829a33 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:22.780137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-754cc6c6-9877-492d-9fd3-ef92687560cc None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:22.781913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-754cc6c6-9877-492d-9fd3-ef92687560cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:22.810753 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cff0b-b968-4139-8bce-073f1190fc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:22.811443 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cff0b-b968-4139-8bce-073f1190fc00 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:22.811725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cff0b-b968-4139-8bce-073f1190fc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:22.811998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cff0b-b968-4139-8bce-073f1190fc00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:22.825453 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cff0b-b968-4139-8bce-073f1190fc00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-a49cff0b-b968-4139-8bce-073f1190fc00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:22.831204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:22.832184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:22.833244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:22.833244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:22.833244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:22.833244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:22.833244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:22.833244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:22.833244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:22.833244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:22.833244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:22.833244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:22.833244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:22.833244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:22.833244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 239/487] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:22.834070 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cff0b-b968-4139-8bce-073f1190fc00 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:22.864566 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 249/488] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:22.866215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cc6c6-9877-492d-9fd3-ef92687560cc None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:22.990091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cc6c6-9877-492d-9fd3-ef92687560cc None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:23.015405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:23.059088 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cff0b-b968-4139-8bce-073f1190fc00 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:23.074596 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c62030-bef5-432c-b409-a6a1fd1fd8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:23.075327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c62030-bef5-432c-b409-a6a1fd1fd8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:23.075610 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c62030-bef5-432c-b409-a6a1fd1fd8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:23.075927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c62030-bef5-432c-b409-a6a1fd1fd8c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:23.076746 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c62030-bef5-432c-b409-a6a1fd1fd8c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:23.077053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c62030-bef5-432c-b409-a6a1fd1fd8c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:23.078478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c62030-bef5-432c-b409-a6a1fd1fd8c7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:23.081303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:23.088975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 250/489] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:23.090454 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d18e0e-9ad2-4bca-835f-200009e5ad4b None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:23.090744 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d18e0e-9ad2-4bca-835f-200009e5ad4b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:23.091928 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d18e0e-9ad2-4bca-835f-200009e5ad4b None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:23.097259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 251/490] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:23.097777 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d18e0e-9ad2-4bca-835f-200009e5ad4b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:23.114932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fec91-661f-4f94-af01-961f762f4ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:23.116081 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fec91-661f-4f94-af01-961f762f4ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:23.116711 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fec91-661f-4f94-af01-961f762f4ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:23.117356 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fec91-661f-4f94-af01-961f762f4ca6 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:23.118505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fec91-661f-4f94-af01-961f762f4ca6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:23.120607 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fec91-661f-4f94-af01-961f762f4ca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:23.123385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fec91-661f-4f94-af01-961f762f4ca6 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:23.152115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 240/491] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:23.152115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8585a5-31f8-4042-a1bd-abe2033d5c43 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:23.152115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8585a5-31f8-4042-a1bd-abe2033d5c43 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:23.152115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8585a5-31f8-4042-a1bd-abe2033d5c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:23.192549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-4c8585a5-31f8-4042-a1bd-abe2033d5c43 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:23.195508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-4c8585a5-31f8-4042-a1bd-abe2033d5c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:23.242054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d18e0e-9ad2-4bca-835f-200009e5ad4b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:23.282386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf706cf-fe34-4455-922c-1bd61c92a91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:23.283053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf706cf-fe34-4455-922c-1bd61c92a91e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:23.283338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf706cf-fe34-4455-922c-1bd61c92a91e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:23.283597 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf706cf-fe34-4455-922c-1bd61c92a91e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:23.296277 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf706cf-fe34-4455-922c-1bd61c92a91e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:23.297811 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf706cf-fe34-4455-922c-1bd61c92a91e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-3cf706cf-fe34-4455-922c-1bd61c92a91e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:23.314860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:23.315889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:23.316786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:23.316786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:23.316786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:23.316786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 252/492] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:23.317023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf706cf-fe34-4455-922c-1bd61c92a91e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:23.346323 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 241/493] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:23.347285 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8585a5-31f8-4042-a1bd-abe2033d5c43 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:23.375090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:23.506013 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8585a5-31f8-4042-a1bd-abe2033d5c43 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:23.507923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:23.521915 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf706cf-fe34-4455-922c-1bd61c92a91e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:23.556565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac653542-2576-4c52-8eb9-9d084317f7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:23.557403 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac653542-2576-4c52-8eb9-9d084317f7c9 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:23.557690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac653542-2576-4c52-8eb9-9d084317f7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:23.557989 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac653542-2576-4c52-8eb9-9d084317f7c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:23.573976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac653542-2576-4c52-8eb9-9d084317f7c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-ac653542-2576-4c52-8eb9-9d084317f7c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:23.580507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:23.582341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:23.583862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:23.583862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:23.583862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:23.583862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:23.583862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:23.583862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:23.583862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:23.583862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:23.583862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:23.583862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:23.583862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:23.583862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:23.587200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 242/494] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:23.588344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac653542-2576-4c52-8eb9-9d084317f7c9 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:23.600434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad83718-fa0c-4e81-b297-664dfb28705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:23.601674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad83718-fa0c-4e81-b297-664dfb28705b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:23.602183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad83718-fa0c-4e81-b297-664dfb28705b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:23.603106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad83718-fa0c-4e81-b297-664dfb28705b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:23.620892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad83718-fa0c-4e81-b297-664dfb28705b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:23.623788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad83718-fa0c-4e81-b297-664dfb28705b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-bad83718-fa0c-4e81-b297-664dfb28705b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:23.645707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:23.646818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:23.647775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:23.647775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:23.647775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:23.650132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 253/495] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:23.651823 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad83718-fa0c-4e81-b297-664dfb28705b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:23.761662 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac653542-2576-4c52-8eb9-9d084317f7c9 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:23.773775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:23.798563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad83718-fa0c-4e81-b297-664dfb28705b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:23.832705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c2d2d-abfd-4344-a454-122b926e601b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:23.833894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c2d2d-abfd-4344-a454-122b926e601b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:23.834401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c2d2d-abfd-4344-a454-122b926e601b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:23.834918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c2d2d-abfd-4344-a454-122b926e601b None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:23.836338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c2d2d-abfd-4344-a454-122b926e601b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:23.836935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c2d2d-abfd-4344-a454-122b926e601b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:23.839747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c2d2d-abfd-4344-a454-122b926e601b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:23.849831 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3571ba35-34bf-4116-9dfd-9874d0f59ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:23.851035 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571ba35-34bf-4116-9dfd-9874d0f59ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:23.851652 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571ba35-34bf-4116-9dfd-9874d0f59ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:23.852841 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571ba35-34bf-4116-9dfd-9874d0f59ec4 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:23.853550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3571ba35-34bf-4116-9dfd-9874d0f59ec4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:23.853859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3571ba35-34bf-4116-9dfd-9874d0f59ec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:23.856045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3571ba35-34bf-4116-9dfd-9874d0f59ec4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:23.864677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 243/496] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:23.871896 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c2d2d-abfd-4344-a454-122b926e601b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:23.872139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 254/497] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:23.872139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3571ba35-34bf-4116-9dfd-9874d0f59ec4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:23.902298 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:23.995934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c2d2d-abfd-4344-a454-122b926e601b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:23.999886 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3571ba35-34bf-4116-9dfd-9874d0f59ec4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:24.039516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301456e0-5075-4d7e-bf0b-25dfd3fbc4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:24.040040 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301456e0-5075-4d7e-bf0b-25dfd3fbc4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:24.040321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301456e0-5075-4d7e-bf0b-25dfd3fbc4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:24.040630 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301456e0-5075-4d7e-bf0b-25dfd3fbc4a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:24.041310 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301456e0-5075-4d7e-bf0b-25dfd3fbc4a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:24.041618 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301456e0-5075-4d7e-bf0b-25dfd3fbc4a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:24.043032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301456e0-5075-4d7e-bf0b-25dfd3fbc4a2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:24.052158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fe613-1110-4682-814c-57c87e93db79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:24.052909 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fe613-1110-4682-814c-57c87e93db79 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:24.053295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fe613-1110-4682-814c-57c87e93db79 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:24.053703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fe613-1110-4682-814c-57c87e93db79 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:24.057045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 255/498] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:24.058477 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1f3b5-939e-4414-a56f-cf504a9456d8 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:24.058880 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1f3b5-939e-4414-a56f-cf504a9456d8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:24.059213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1f3b5-939e-4414-a56f-cf504a9456d8 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:24.061027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 256/499] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:24.065477 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1f3b5-939e-4414-a56f-cf504a9456d8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:24.070651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742fe613-1110-4682-814c-57c87e93db79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-742fe613-1110-4682-814c-57c87e93db79 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:24.076634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:24.078349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:24.079943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:24.079943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:24.079943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:24.079943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:24.079943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:24.079943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:24.079943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:24.079943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:24.079943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:24.079943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:24.079943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:24.079943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:24.079943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 244/500] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:24.079943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fe613-1110-4682-814c-57c87e93db79 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:24.082567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:24.180631 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:24.202388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fe613-1110-4682-814c-57c87e93db79 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:24.245498 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1f3b5-939e-4414-a56f-cf504a9456d8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:24.245952 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e343900-c813-40b4-b1e2-2871bccaa8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:24.246604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e343900-c813-40b4-b1e2-2871bccaa8bb None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:24.246884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e343900-c813-40b4-b1e2-2871bccaa8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:24.247157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e343900-c813-40b4-b1e2-2871bccaa8bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:24.259209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e343900-c813-40b4-b1e2-2871bccaa8bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:24.260734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e343900-c813-40b4-b1e2-2871bccaa8bb None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-6e343900-c813-40b4-b1e2-2871bccaa8bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:24.278316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:24.280129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:24.281943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:24.281943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:24.281943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:24.281943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 245/501] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:24.281943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e343900-c813-40b4-b1e2-2871bccaa8bb None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:24.312473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873a077-8bc0-419b-ad15-513b608235b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:24.313468 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873a077-8bc0-419b-ad15-513b608235b3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:24.313958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873a077-8bc0-419b-ad15-513b608235b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:24.314445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873a077-8bc0-419b-ad15-513b608235b3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:24.328899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873a077-8bc0-419b-ad15-513b608235b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:24.330808 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873a077-8bc0-419b-ad15-513b608235b3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-a873a077-8bc0-419b-ad15-513b608235b3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:24.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:24.347405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:24.349298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:24.349298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:24.349298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:24.349298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:24.349298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:24.349298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:24.349298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:24.349298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:24.349298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:28:24.349298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:24.349298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 257/502] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:24.349298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a9f0c-c310-4e95-8306-76ee93248ad6 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:24.349298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a9f0c-c310-4e95-8306-76ee93248ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:24.350578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a9f0c-c310-4e95-8306-76ee93248ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:24.368191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3c1a9f0c-c310-4e95-8306-76ee93248ad6 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:24.369486 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3c1a9f0c-c310-4e95-8306-76ee93248ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:24.440950 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:24.443122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e343900-c813-40b4-b1e2-2871bccaa8bb None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:24.449120 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 258/503] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:24.449856 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a9f0c-c310-4e95-8306-76ee93248ad6 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:24.497459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee4483-ae66-4010-bf2a-07dbeda92d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:24.498679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee4483-ae66-4010-bf2a-07dbeda92d7c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:24.498962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee4483-ae66-4010-bf2a-07dbeda92d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:24.499242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee4483-ae66-4010-bf2a-07dbeda92d7c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:24.501809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:24.510696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee4483-ae66-4010-bf2a-07dbeda92d7c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:24.511092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee4483-ae66-4010-bf2a-07dbeda92d7c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-1fee4483-ae66-4010-bf2a-07dbeda92d7c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:24.525409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:24.526412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:24.527519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:24.527519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:24.527519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:24.527519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:24.527519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:24.527519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:24.527519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:24.527519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:24.527519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:28:24.527519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:24.527519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 246/504] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:24.527519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee4483-ae66-4010-bf2a-07dbeda92d7c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:24.576049 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a9f0c-c310-4e95-8306-76ee93248ad6 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:24.615116 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220139c5-cc91-4299-812c-6528e807ff71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:24.615983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220139c5-cc91-4299-812c-6528e807ff71 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:24.616438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220139c5-cc91-4299-812c-6528e807ff71 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:24.616871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220139c5-cc91-4299-812c-6528e807ff71 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:24.617718 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220139c5-cc91-4299-812c-6528e807ff71 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:24.618165 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220139c5-cc91-4299-812c-6528e807ff71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:24.619703 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220139c5-cc91-4299-812c-6528e807ff71 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:24.629989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 259/505] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:24.631095 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220139c5-cc91-4299-812c-6528e807ff71 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:24.693422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee4483-ae66-4010-bf2a-07dbeda92d7c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:24.720746 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:24.779990 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c8088-8f9f-45e8-a591-c324d9fa9d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:24.780643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c8088-8f9f-45e8-a591-c324d9fa9d71 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:24.780927 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c8088-8f9f-45e8-a591-c324d9fa9d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:24.781209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c8088-8f9f-45e8-a591-c324d9fa9d71 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:24.781905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c8088-8f9f-45e8-a591-c324d9fa9d71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:24.782344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c8088-8f9f-45e8-a591-c324d9fa9d71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:24.783603 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c8088-8f9f-45e8-a591-c324d9fa9d71 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:24.790597 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220139c5-cc91-4299-812c-6528e807ff71 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:24.811458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 247/506] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:24.811458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c8088-8f9f-45e8-a591-c324d9fa9d71 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:24.840962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e026b-f164-43fa-ae77-7be98d5acfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:24.842144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e026b-f164-43fa-ae77-7be98d5acfca None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:24.842520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e026b-f164-43fa-ae77-7be98d5acfca None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:24.842896 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e026b-f164-43fa-ae77-7be98d5acfca None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:24.843782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e026b-f164-43fa-ae77-7be98d5acfca None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:24.844199 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e026b-f164-43fa-ae77-7be98d5acfca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:24.845859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e026b-f164-43fa-ae77-7be98d5acfca None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:24.894405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 260/507] 10.222.0.70 () {56 vars in 1137 bytes} [Wed Aug 24 16:28:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:24.895026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:24.895144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e026b-f164-43fa-ae77-7be98d5acfca None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:24.962729 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c8088-8f9f-45e8-a591-c324d9fa9d71 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:25.016385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2827a4-025d-4350-a5d6-a757e0391c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:25.017237 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2827a4-025d-4350-a5d6-a757e0391c4f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:25.017689 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2827a4-025d-4350-a5d6-a757e0391c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:25.018133 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2827a4-025d-4350-a5d6-a757e0391c4f None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:25.019357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2827a4-025d-4350-a5d6-a757e0391c4f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:25.019845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2827a4-025d-4350-a5d6-a757e0391c4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:25.022363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2827a4-025d-4350-a5d6-a757e0391c4f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:25.053570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e026b-f164-43fa-ae77-7be98d5acfca None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:25.073547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:25.074855 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 248/508] 10.222.0.70 () {56 vars in 1137 bytes} [Wed Aug 24 16:28:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:25.076352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2827a4-025d-4350-a5d6-a757e0391c4f None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:25.095200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3fc8-e278-4470-834b-ccaff6884a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:25.095773 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3fc8-e278-4470-834b-ccaff6884a41 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:25.096030 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3fc8-e278-4470-834b-ccaff6884a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:25.096349 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3fc8-e278-4470-834b-ccaff6884a41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:25.108733 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c3fc8-e278-4470-834b-ccaff6884a41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:25.110505 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c3fc8-e278-4470-834b-ccaff6884a41 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-de0c3fc8-e278-4470-834b-ccaff6884a41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:25.126719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:25.131500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:25.133235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:25.133235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:25.133235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:25.133235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 261/509] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:25.133235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3fc8-e278-4470-834b-ccaff6884a41 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:25.212663 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2827a4-025d-4350-a5d6-a757e0391c4f None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:25.233123 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:25.255746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a191b68-27f2-4cc3-8a58-4da8a1fff382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:25.256391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a191b68-27f2-4cc3-8a58-4da8a1fff382 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:25.256753 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a191b68-27f2-4cc3-8a58-4da8a1fff382 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:25.256966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a191b68-27f2-4cc3-8a58-4da8a1fff382 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:25.265388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a191b68-27f2-4cc3-8a58-4da8a1fff382 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:25.267127 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a191b68-27f2-4cc3-8a58-4da8a1fff382 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:25.268642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3fc8-e278-4470-834b-ccaff6884a41 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-1a191b68-27f2-4cc3-8a58-4da8a1fff382 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:25.282385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:25.283494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:25.285569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:25.285569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:25.285569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:25.285569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:25.285569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:25.285569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:25.285569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:25.285569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:25.285569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:28:25.285569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:25.285569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 249/510] 10.222.0.70 () {56 vars in 1093 bytes} [Wed Aug 24 16:28:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:25.285569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a191b68-27f2-4cc3-8a58-4da8a1fff382 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:25.329295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a42d9e-8a9d-479a-84ae-b9ed238d39a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:25.329920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a42d9e-8a9d-479a-84ae-b9ed238d39a7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:25.330205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a42d9e-8a9d-479a-84ae-b9ed238d39a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:25.330478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a42d9e-8a9d-479a-84ae-b9ed238d39a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:25.389543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:25.412587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a191b68-27f2-4cc3-8a58-4da8a1fff382 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:25.446290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a42d9e-8a9d-479a-84ae-b9ed238d39a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:25.448362 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a42d9e-8a9d-479a-84ae-b9ed238d39a7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:25.454055 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cc02b-898a-45c2-bc9a-3774044a40d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:25.454877 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cc02b-898a-45c2-bc9a-3774044a40d8 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:25.455269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cc02b-898a-45c2-bc9a-3774044a40d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:25.455665 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cc02b-898a-45c2-bc9a-3774044a40d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:25.470412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4cc02b-898a-45c2-bc9a-3774044a40d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:25.473071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4cc02b-898a-45c2-bc9a-3774044a40d8 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-ff4cc02b-898a-45c2-bc9a-3774044a40d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:25.495135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:25.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:25.498146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:25.498146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:25.498146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:25.498146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 250/511] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:25.498146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cc02b-898a-45c2-bc9a-3774044a40d8 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:25.541773 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:25.583381 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 262/512] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:25.583381 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a42d9e-8a9d-479a-84ae-b9ed238d39a7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:25.635930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cc02b-898a-45c2-bc9a-3774044a40d8 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:25.677913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:25.679994 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7079049-25ad-4cae-9f35-274191497e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:25.680770 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7079049-25ad-4cae-9f35-274191497e81 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:25.681151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7079049-25ad-4cae-9f35-274191497e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:25.681530 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7079049-25ad-4cae-9f35-274191497e81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:25.725431 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a42d9e-8a9d-479a-84ae-b9ed238d39a7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:25.768990 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7079049-25ad-4cae-9f35-274191497e81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:25.770110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49dcea6-60ac-4a1f-a845-cfeed6ded1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:25.770954 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7079049-25ad-4cae-9f35-274191497e81 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:25.771121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49dcea6-60ac-4a1f-a845-cfeed6ded1db None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:25.771488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49dcea6-60ac-4a1f-a845-cfeed6ded1db None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:25.771923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49dcea6-60ac-4a1f-a845-cfeed6ded1db None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:25.773047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49dcea6-60ac-4a1f-a845-cfeed6ded1db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:25.773544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49dcea6-60ac-4a1f-a845-cfeed6ded1db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:25.775954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49dcea6-60ac-4a1f-a845-cfeed6ded1db None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:25.796755 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 263/513] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:25.797740 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49dcea6-60ac-4a1f-a845-cfeed6ded1db None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:25.800614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:25.900321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 251/514] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:25.901942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7079049-25ad-4cae-9f35-274191497e81 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:25.933844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49dcea6-60ac-4a1f-a845-cfeed6ded1db None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:25.969098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:25.980563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3348a-eac1-4faa-8086-f1b281bf68b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:25.981537 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3348a-eac1-4faa-8086-f1b281bf68b9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:25.982504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3348a-eac1-4faa-8086-f1b281bf68b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:25.982836 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3348a-eac1-4faa-8086-f1b281bf68b9 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:25.984080 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3348a-eac1-4faa-8086-f1b281bf68b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:25.984614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3348a-eac1-4faa-8086-f1b281bf68b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:25.987151 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3348a-eac1-4faa-8086-f1b281bf68b9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:26.015798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 264/515] 10.222.0.70 () {56 vars in 1133 bytes} [Wed Aug 24 16:28:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:26.015798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3348a-eac1-4faa-8086-f1b281bf68b9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:26.046972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7079049-25ad-4cae-9f35-274191497e81 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:26.081211 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:26.097296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14365ae0-fbb2-4aba-a2d7-2e413935c17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:26.098224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14365ae0-fbb2-4aba-a2d7-2e413935c17d None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:26.098643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14365ae0-fbb2-4aba-a2d7-2e413935c17d None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:26.099094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14365ae0-fbb2-4aba-a2d7-2e413935c17d None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:26.100204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14365ae0-fbb2-4aba-a2d7-2e413935c17d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:26.100747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14365ae0-fbb2-4aba-a2d7-2e413935c17d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:26.103270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14365ae0-fbb2-4aba-a2d7-2e413935c17d None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:26.123884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 252/516] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:26.124945 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14365ae0-fbb2-4aba-a2d7-2e413935c17d None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:26.168479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3348a-eac1-4faa-8086-f1b281bf68b9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:26.216517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1624d-269a-43a0-9399-ac132dc75b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:26.217188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1624d-269a-43a0-9399-ac132dc75b9b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:26.217541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1624d-269a-43a0-9399-ac132dc75b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:26.217901 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1624d-269a-43a0-9399-ac132dc75b9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:26.230851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1624d-269a-43a0-9399-ac132dc75b9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:26.233044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1624d-269a-43a0-9399-ac132dc75b9b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-57d1624d-269a-43a0-9399-ac132dc75b9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:26.252335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:26.254147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:26.256127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:26.256127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:26.256127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:26.256127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 265/517] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:26.258827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1624d-269a-43a0-9399-ac132dc75b9b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:26.266569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14365ae0-fbb2-4aba-a2d7-2e413935c17d None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:26.307484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:26.313540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423f476-811d-40f8-a98f-a7dc9fc0dba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:26.314317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423f476-811d-40f8-a98f-a7dc9fc0dba2 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:26.314687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423f476-811d-40f8-a98f-a7dc9fc0dba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:26.315049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423f476-811d-40f8-a98f-a7dc9fc0dba2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:26.325173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423f476-811d-40f8-a98f-a7dc9fc0dba2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:26.326711 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423f476-811d-40f8-a98f-a7dc9fc0dba2 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-b423f476-811d-40f8-a98f-a7dc9fc0dba2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:26.346136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:26.347253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:26.348748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:26.348748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:26.348748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:26.348748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:26.348748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:26.348748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:26.348748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:26.348748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:26.348748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:28:26.348748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:26.348748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 253/518] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:26.351238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423f476-811d-40f8-a98f-a7dc9fc0dba2 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:26.389157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1624d-269a-43a0-9399-ac132dc75b9b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:26.409121 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:26.428665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec3dee-0d8f-40e6-8dc7-f4b36bf2ecfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:26.429325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec3dee-0d8f-40e6-8dc7-f4b36bf2ecfa None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:26.429608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec3dee-0d8f-40e6-8dc7-f4b36bf2ecfa None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:26.429878 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec3dee-0d8f-40e6-8dc7-f4b36bf2ecfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:26.442380 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec3dee-0d8f-40e6-8dc7-f4b36bf2ecfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:26.444067 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec3dee-0d8f-40e6-8dc7-f4b36bf2ecfa None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-5cec3dee-0d8f-40e6-8dc7-f4b36bf2ecfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:26.459882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:26.461078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:26.463222 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:26.463222 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:26.463222 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:26.463222 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 266/519] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:26.463222 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec3dee-0d8f-40e6-8dc7-f4b36bf2ecfa None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:26.505469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423f476-811d-40f8-a98f-a7dc9fc0dba2 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:26.564056 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac31ee8-84af-4672-abc0-8549a4c24d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:26.564971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac31ee8-84af-4672-abc0-8549a4c24d3e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:26.565478 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac31ee8-84af-4672-abc0-8549a4c24d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:26.565919 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac31ee8-84af-4672-abc0-8549a4c24d3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:26.583051 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:26.598648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec3dee-0d8f-40e6-8dc7-f4b36bf2ecfa None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:26.655324 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2924b8-14ea-4d40-b39d-4ed1d5f094d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:26.656271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2924b8-14ea-4d40-b39d-4ed1d5f094d3 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:26.656771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2924b8-14ea-4d40-b39d-4ed1d5f094d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:26.657205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2924b8-14ea-4d40-b39d-4ed1d5f094d3 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:26.657971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2924b8-14ea-4d40-b39d-4ed1d5f094d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:26.659642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2924b8-14ea-4d40-b39d-4ed1d5f094d3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:26.674042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac31ee8-84af-4672-abc0-8549a4c24d3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:26.675589 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac31ee8-84af-4672-abc0-8549a4c24d3e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:26.730797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:26.762526 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 267/520] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:28:26 2022] POST /identity/v3/users => generated 247 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:28:26.764530 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2924b8-14ea-4d40-b39d-4ed1d5f094d3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:26.831716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 254/521] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:26.832847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac31ee8-84af-4672-abc0-8549a4c24d3e None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:26.955432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2924b8-14ea-4d40-b39d-4ed1d5f094d3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:27.002522 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac31ee8-84af-4672-abc0-8549a4c24d3e None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:27.014443 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ca0ef-17b7-4391-9577-b73e73088210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:27.014976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ca0ef-17b7-4391-9577-b73e73088210 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:27.015259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ca0ef-17b7-4391-9577-b73e73088210 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:27.015513 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ca0ef-17b7-4391-9577-b73e73088210 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:27.016565 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ca0ef-17b7-4391-9577-b73e73088210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:27.017411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ca0ef-17b7-4391-9577-b73e73088210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:27.018453 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ca0ef-17b7-4391-9577-b73e73088210 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:27.041590 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 268/522] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:27.043234 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ca0ef-17b7-4391-9577-b73e73088210 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:27.058486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e54061-f268-4755-b009-2bac300a90d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:27.060126 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:27.061192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e54061-f268-4755-b009-2bac300a90d4 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:27.061667 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e54061-f268-4755-b009-2bac300a90d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:27.062323 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e54061-f268-4755-b009-2bac300a90d4 None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:27.063783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e54061-f268-4755-b009-2bac300a90d4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:27.064355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e54061-f268-4755-b009-2bac300a90d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:27.066818 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e54061-f268-4755-b009-2bac300a90d4 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:27.099021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 255/523] 10.222.0.70 () {56 vars in 1139 bytes} [Wed Aug 24 16:28:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:27.100091 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e54061-f268-4755-b009-2bac300a90d4 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:27.103484 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:27.194165 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ca0ef-17b7-4391-9577-b73e73088210 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:27.254596 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6718ba8-0a44-414e-b830-5bd8e3674357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:27.255216 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6718ba8-0a44-414e-b830-5bd8e3674357 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:27.255518 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6718ba8-0a44-414e-b830-5bd8e3674357 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:27.255758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6718ba8-0a44-414e-b830-5bd8e3674357 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:27.256468 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6718ba8-0a44-414e-b830-5bd8e3674357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:27.256864 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6718ba8-0a44-414e-b830-5bd8e3674357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:27.258322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6718ba8-0a44-414e-b830-5bd8e3674357 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:27.258514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e54061-f268-4755-b009-2bac300a90d4 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:27.277275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 269/524] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:27.277275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6718ba8-0a44-414e-b830-5bd8e3674357 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:27.329539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672b6f9-2893-42ba-8364-df66137338af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:27.330589 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672b6f9-2893-42ba-8364-df66137338af None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:27.330990 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672b6f9-2893-42ba-8364-df66137338af None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:27.331370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672b6f9-2893-42ba-8364-df66137338af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:27.349026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672b6f9-2893-42ba-8364-df66137338af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:27.351587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672b6f9-2893-42ba-8364-df66137338af None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:27.359888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-d672b6f9-2893-42ba-8364-df66137338af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:27.370612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:27.375923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:27.377644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:27.377644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:27.377644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:27.377644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 256/525] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:27.377930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672b6f9-2893-42ba-8364-df66137338af None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:27.426831 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6718ba8-0a44-414e-b830-5bd8e3674357 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:27.454308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:27.487746 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047278b4-a28a-413b-9a4c-974ab9cfd5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:27.488532 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047278b4-a28a-413b-9a4c-974ab9cfd5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:27.488810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047278b4-a28a-413b-9a4c-974ab9cfd5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:27.489082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047278b4-a28a-413b-9a4c-974ab9cfd5e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:27.499079 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047278b4-a28a-413b-9a4c-974ab9cfd5e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:27.500601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047278b4-a28a-413b-9a4c-974ab9cfd5e2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:27.514325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-047278b4-a28a-413b-9a4c-974ab9cfd5e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:27.514325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:27.514325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:27.514325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:27.514325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:27.514325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:27.514325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:27.514325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:27.514325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:27.514325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:27.514325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:27.514325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:27.514325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:27.514325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:27.514325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:27.516541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:27.518475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:27.518475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:27.518475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:27.518475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:27.518475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:27.518475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:27.518475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:27.518475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:27.518475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 270/526] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:27.518475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047278b4-a28a-413b-9a4c-974ab9cfd5e2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:27.544275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672b6f9-2893-42ba-8364-df66137338af None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:27.604515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32553bb1-1884-4cad-8cba-35816a1d730e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:27.605539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32553bb1-1884-4cad-8cba-35816a1d730e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:27.606023 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32553bb1-1884-4cad-8cba-35816a1d730e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:27.606474 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32553bb1-1884-4cad-8cba-35816a1d730e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:27.700875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047278b4-a28a-413b-9a4c-974ab9cfd5e2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:27.712072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32553bb1-1884-4cad-8cba-35816a1d730e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:27.713924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32553bb1-1884-4cad-8cba-35816a1d730e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:27.745968 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:27.760722 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6c76a-0dac-4534-8f70-58fc160d0ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:27.762465 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6c76a-0dac-4534-8f70-58fc160d0ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:27.763322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6c76a-0dac-4534-8f70-58fc160d0ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:27.763762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6c76a-0dac-4534-8f70-58fc160d0ce1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:27.774992 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6c76a-0dac-4534-8f70-58fc160d0ce1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:27.776634 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6c76a-0dac-4534-8f70-58fc160d0ce1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:27.790227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f6e6c76a-0dac-4534-8f70-58fc160d0ce1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:27.790227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:27.790227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:27.790227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:27.790227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:27.790227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:27.790227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:27.790227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:27.790227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:27.790227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:27.790227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:27.790227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:27.790227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:27.790227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:27.790227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:27.792208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:27.794306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:27.794306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:27.794306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:27.794306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:27.794306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:27.794306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:27.794306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:27.794306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:27.803125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 271/527] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:27.804372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6c76a-0dac-4534-8f70-58fc160d0ce1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:27.825940 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:27.911896 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 257/528] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:27.913756 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32553bb1-1884-4cad-8cba-35816a1d730e None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:27.975663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6c76a-0dac-4534-8f70-58fc160d0ce1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:28.034784 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af62a8-1616-4caa-8501-fdde9a10cae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:28.035403 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af62a8-1616-4caa-8501-fdde9a10cae4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:28.035677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af62a8-1616-4caa-8501-fdde9a10cae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:28.036117 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af62a8-1616-4caa-8501-fdde9a10cae4 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:28.036812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af62a8-1616-4caa-8501-fdde9a10cae4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:28.037125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af62a8-1616-4caa-8501-fdde9a10cae4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:28.038561 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af62a8-1616-4caa-8501-fdde9a10cae4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:28.063370 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 272/529] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:28.063370 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af62a8-1616-4caa-8501-fdde9a10cae4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:28.090175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32553bb1-1884-4cad-8cba-35816a1d730e None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:28.145116 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:28.145380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbcfc7-dcce-4190-8dac-8def3c3c13fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:28.145984 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbcfc7-dcce-4190-8dac-8def3c3c13fd None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:28.146197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbcfc7-dcce-4190-8dac-8def3c3c13fd None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:28.146471 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbcfc7-dcce-4190-8dac-8def3c3c13fd None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:28.147185 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbcfc7-dcce-4190-8dac-8def3c3c13fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:28.147613 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbcfc7-dcce-4190-8dac-8def3c3c13fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:28.149662 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbcfc7-dcce-4190-8dac-8def3c3c13fd None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:28.177710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 258/530] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:28.179560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbcfc7-dcce-4190-8dac-8def3c3c13fd None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:28.207383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:28.231717 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af62a8-1616-4caa-8501-fdde9a10cae4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:28.274916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc9f0f-4159-4a8d-8fd6-135b5d49a749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:28.275548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc9f0f-4159-4a8d-8fd6-135b5d49a749 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:28.275828 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc9f0f-4159-4a8d-8fd6-135b5d49a749 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:28.276097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc9f0f-4159-4a8d-8fd6-135b5d49a749 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:28.276851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc9f0f-4159-4a8d-8fd6-135b5d49a749 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:28.277163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc9f0f-4159-4a8d-8fd6-135b5d49a749 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:28.278686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc9f0f-4159-4a8d-8fd6-135b5d49a749 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:28.304854 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 273/531] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:28.306767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc9f0f-4159-4a8d-8fd6-135b5d49a749 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:28.347992 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbcfc7-dcce-4190-8dac-8def3c3c13fd None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:28.404634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1b0f0-5c6c-4179-b230-ae74b0cb89bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:28.405296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1b0f0-5c6c-4179-b230-ae74b0cb89bb None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:28.405577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1b0f0-5c6c-4179-b230-ae74b0cb89bb None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:28.405847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1b0f0-5c6c-4179-b230-ae74b0cb89bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:28.418217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b1b0f0-5c6c-4179-b230-ae74b0cb89bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:28.420220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b1b0f0-5c6c-4179-b230-ae74b0cb89bb None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-c4b1b0f0-5c6c-4179-b230-ae74b0cb89bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:28.437899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:28.439320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:28.440678 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:28.440678 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:28.440678 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:28.440678 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 259/532] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:28.444332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1b0f0-5c6c-4179-b230-ae74b0cb89bb None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:28.446837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc9f0f-4159-4a8d-8fd6-135b5d49a749 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:28.466467 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:28.509970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1719a3-2191-4123-b075-f85729da67f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:28.510620 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1719a3-2191-4123-b075-f85729da67f2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:28.510902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1719a3-2191-4123-b075-f85729da67f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:28.511176 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1719a3-2191-4123-b075-f85729da67f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:28.520891 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1719a3-2191-4123-b075-f85729da67f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:28.522400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1719a3-2191-4123-b075-f85729da67f2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:28.524315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:28.543888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-8f1719a3-2191-4123-b075-f85729da67f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:28.543888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:28.543888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:28.543888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:28.543888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:28.543888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:28.543888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:28.543888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:28.543888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:28.543888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:28.543888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:28.543888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:28.543888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:28.543888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:28.543888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:28.546373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:28.547391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:28.547391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:28.547391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:28.547391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:28.547391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:28.547391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:28.547391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:28.547391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:28.549577 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 274/533] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:28.551309 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1719a3-2191-4123-b075-f85729da67f2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:28.580694 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1b0f0-5c6c-4179-b230-ae74b0cb89bb None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:28.627434 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0202b5-8760-4d51-89dd-eb97c280841b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:28.628556 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0202b5-8760-4d51-89dd-eb97c280841b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:28.628944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0202b5-8760-4d51-89dd-eb97c280841b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:28.629327 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0202b5-8760-4d51-89dd-eb97c280841b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:28.630077 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0202b5-8760-4d51-89dd-eb97c280841b None admin] RBAC: Adding query filter params (role.id=bb34a44869834493b10a99cc751ab6c2, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=56418ff737504181a59714aad9f1c375) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:28.630390 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0202b5-8760-4d51-89dd-eb97c280841b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:28.631858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0202b5-8760-4d51-89dd-eb97c280841b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:28.638865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 260/534] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:28 2022] GET /identity/v3/role_assignments?user.id=56418ff737504181a59714aad9f1c375&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=bb34a44869834493b10a99cc751ab6c2 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:28.641923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0202b5-8760-4d51-89dd-eb97c280841b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:28.714722 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1719a3-2191-4123-b075-f85729da67f2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:28.765257 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcacda-f16e-42b7-978d-5c91c894a2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:28.766419 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcacda-f16e-42b7-978d-5c91c894a2cb None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:28.766834 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcacda-f16e-42b7-978d-5c91c894a2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:28.767243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcacda-f16e-42b7-978d-5c91c894a2cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:28.768143 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcacda-f16e-42b7-978d-5c91c894a2cb None admin] RBAC: Adding query filter params (role.id=bb34a44869834493b10a99cc751ab6c2, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=f08140bcbd1e43dda77d4387666f9085) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:28.768681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcacda-f16e-42b7-978d-5c91c894a2cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:28.770341 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcacda-f16e-42b7-978d-5c91c894a2cb None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:28.777029 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 275/535] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:28 2022] GET /identity/v3/role_assignments?user.id=f08140bcbd1e43dda77d4387666f9085&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=bb34a44869834493b10a99cc751ab6c2 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:28.778138 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcacda-f16e-42b7-978d-5c91c894a2cb None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:28.811962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0202b5-8760-4d51-89dd-eb97c280841b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:28.828175 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:28.866580 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:28.879178 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93ee69-e869-4cf5-9ea4-872c768d8ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:28.881022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93ee69-e869-4cf5-9ea4-872c768d8ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:28.882050 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93ee69-e869-4cf5-9ea4-872c768d8ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:28.882901 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93ee69-e869-4cf5-9ea4-872c768d8ab9 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:28.884915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93ee69-e869-4cf5-9ea4-872c768d8ab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:28.885839 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93ee69-e869-4cf5-9ea4-872c768d8ab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:28.889437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93ee69-e869-4cf5-9ea4-872c768d8ab9 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:28.915768 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 261/536] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:28.922478 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93ee69-e869-4cf5-9ea4-872c768d8ab9 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:28.930287 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcacda-f16e-42b7-978d-5c91c894a2cb None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:28.998008 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e67a9-1c94-4d39-bc3a-d9eacae19c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:28.998986 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e67a9-1c94-4d39-bc3a-d9eacae19c92 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:28.999880 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e67a9-1c94-4d39-bc3a-d9eacae19c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:29.000296 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e67a9-1c94-4d39-bc3a-d9eacae19c92 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:29.001441 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e67a9-1c94-4d39-bc3a-d9eacae19c92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:29.001888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e67a9-1c94-4d39-bc3a-d9eacae19c92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:29.004107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e67a9-1c94-4d39-bc3a-d9eacae19c92 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:29.025363 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 276/537] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:29.027594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b88c2f-130a-4d76-a137-912baeab948f None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:29.028153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b88c2f-130a-4d76-a137-912baeab948f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:29.028694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b88c2f-130a-4d76-a137-912baeab948f None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:29.031777 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 277/538] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:29.040455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c95c0-573e-42b4-8053-249760fa9774 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:29.040929 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c95c0-573e-42b4-8053-249760fa9774 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:29.041397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c95c0-573e-42b4-8053-249760fa9774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:29.073273 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-278c95c0-573e-42b4-8053-249760fa9774 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:29.075388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-278c95c0-573e-42b4-8053-249760fa9774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:29.085656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93ee69-e869-4cf5-9ea4-872c768d8ab9 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:29.139343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ce2b6-b913-4d44-9c62-712d4b7df122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:29.140323 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ce2b6-b913-4d44-9c62-712d4b7df122 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:29.140880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ce2b6-b913-4d44-9c62-712d4b7df122 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:29.141640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ce2b6-b913-4d44-9c62-712d4b7df122 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:29.155523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ce2b6-b913-4d44-9c62-712d4b7df122 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:29.158202 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ce2b6-b913-4d44-9c62-712d4b7df122 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:29.174281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 278/539] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:28:29.175204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c95c0-573e-42b4-8053-249760fa9774 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:29.193560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-543ce2b6-b913-4d44-9c62-712d4b7df122 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:29.193560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:29.193560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:29.193560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:29.193560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:29.193560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:29.193560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:29.193560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:29.193560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:29.193560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:29.193560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:29.193560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:29.193560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:29.193560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:29.193560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:29.195222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:29.196809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:29.196809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:29.196809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:29.196809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:29.196809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:29.196809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:29.196809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:29.196809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:29.196809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 262/540] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:29.201746 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:29.207983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ce2b6-b913-4d44-9c62-712d4b7df122 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:29.395573 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:29.399012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c95c0-573e-42b4-8053-249760fa9774 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:29.437956 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613f10c-c395-41b9-b70c-6bd6eafaf035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:29.439107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613f10c-c395-41b9-b70c-6bd6eafaf035 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:29.439597 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613f10c-c395-41b9-b70c-6bd6eafaf035 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:29.440226 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613f10c-c395-41b9-b70c-6bd6eafaf035 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:29.441647 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613f10c-c395-41b9-b70c-6bd6eafaf035 None admin] RBAC: Adding query filter params (role.id=bb34a44869834493b10a99cc751ab6c2, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=e333fca7790244f6a4ebceabb55faba4) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:29.442209 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613f10c-c395-41b9-b70c-6bd6eafaf035 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:29.445091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613f10c-c395-41b9-b70c-6bd6eafaf035 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:29.455486 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 279/541] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:29 2022] GET /identity/v3/role_assignments?user.id=e333fca7790244f6a4ebceabb55faba4&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=bb34a44869834493b10a99cc751ab6c2 => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:29.525131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ce2b6-b913-4d44-9c62-712d4b7df122 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:29.592253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc344d-1284-4bab-b301-c64299379322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:29.593338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc344d-1284-4bab-b301-c64299379322 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:29.593807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc344d-1284-4bab-b301-c64299379322 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:29.594225 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc344d-1284-4bab-b301-c64299379322 None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:29.595367 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc344d-1284-4bab-b301-c64299379322 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:29.595920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc344d-1284-4bab-b301-c64299379322 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:29.598671 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc344d-1284-4bab-b301-c64299379322 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:29.624386 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 263/542] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:29.633792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613f10c-c395-41b9-b70c-6bd6eafaf035 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:29.703970 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc344d-1284-4bab-b301-c64299379322 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:29.760938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:29.775413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613f10c-c395-41b9-b70c-6bd6eafaf035 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:29.817175 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f14f5-cd26-482f-92f3-ce6bd449ddd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:29.817799 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f14f5-cd26-482f-92f3-ce6bd449ddd3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:29.818072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f14f5-cd26-482f-92f3-ce6bd449ddd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:29.818343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f14f5-cd26-482f-92f3-ce6bd449ddd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:29.819087 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f14f5-cd26-482f-92f3-ce6bd449ddd3 None admin] RBAC: Adding query filter params (role.id=bb34a44869834493b10a99cc751ab6c2, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=942b54c39ae5403c8ae3bcca1c31bf1a) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:29.819390 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f14f5-cd26-482f-92f3-ce6bd449ddd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:29.820973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f14f5-cd26-482f-92f3-ce6bd449ddd3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:29.827898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 280/543] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:29 2022] GET /identity/v3/role_assignments?user.id=942b54c39ae5403c8ae3bcca1c31bf1a&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=bb34a44869834493b10a99cc751ab6c2 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:29.958263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc344d-1284-4bab-b301-c64299379322 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:29.980788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:30.031056 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864be140-39ed-4339-9f31-750c023a1d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:30.031951 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864be140-39ed-4339-9f31-750c023a1d07 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:30.032349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864be140-39ed-4339-9f31-750c023a1d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:30.032774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864be140-39ed-4339-9f31-750c023a1d07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:30.061781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864be140-39ed-4339-9f31-750c023a1d07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-864be140-39ed-4339-9f31-750c023a1d07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:30.070598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:30.072348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:30.073980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:30.073980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:30.073980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:30.073980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:30.073980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:30.073980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:30.073980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:30.073980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:30.073980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:30.073980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:30.073980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:30.073980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:30.073980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 264/544] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:28:30.081493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f14f5-cd26-482f-92f3-ce6bd449ddd3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:30.292548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f14f5-cd26-482f-92f3-ce6bd449ddd3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:30.350447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd9f95-edca-4d94-a901-08813bcb2fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:30.351194 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd9f95-edca-4d94-a901-08813bcb2fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:30.351476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd9f95-edca-4d94-a901-08813bcb2fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:30.351752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd9f95-edca-4d94-a901-08813bcb2fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:30.352476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd9f95-edca-4d94-a901-08813bcb2fc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:30.352785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd9f95-edca-4d94-a901-08813bcb2fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:30.354208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd9f95-edca-4d94-a901-08813bcb2fc2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:30.369747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 281/545] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:30.376726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864be140-39ed-4339-9f31-750c023a1d07 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:30.588775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864be140-39ed-4339-9f31-750c023a1d07 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:30.632284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a00de-b9c5-45d9-b3c6-a0944dbf37fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:30.632988 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a00de-b9c5-45d9-b3c6-a0944dbf37fc None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:30.633268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a00de-b9c5-45d9-b3c6-a0944dbf37fc None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:30.633540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a00de-b9c5-45d9-b3c6-a0944dbf37fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:30.646935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a00de-b9c5-45d9-b3c6-a0944dbf37fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:30.652547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a00de-b9c5-45d9-b3c6-a0944dbf37fc None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-f93a00de-b9c5-45d9-b3c6-a0944dbf37fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:30.675204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:30.676965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:30.680830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:30.680830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:30.680830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:30.680830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 265/546] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:28:30.684587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd9f95-edca-4d94-a901-08813bcb2fc2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:30.710798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:30.916069 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd9f95-edca-4d94-a901-08813bcb2fc2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:30.985711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190910a-9a2c-409a-a6f2-1ddd3b1dcee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:30.986721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190910a-9a2c-409a-a6f2-1ddd3b1dcee7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:30.987221 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190910a-9a2c-409a-a6f2-1ddd3b1dcee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:30.987715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190910a-9a2c-409a-a6f2-1ddd3b1dcee7 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:30.989000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190910a-9a2c-409a-a6f2-1ddd3b1dcee7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:30.989548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190910a-9a2c-409a-a6f2-1ddd3b1dcee7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:30.992281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190910a-9a2c-409a-a6f2-1ddd3b1dcee7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:31.017753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 282/547] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:31.023386 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a00de-b9c5-45d9-b3c6-a0944dbf37fc None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:31.074383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:31.196323 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a00de-b9c5-45d9-b3c6-a0944dbf37fc None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:31.246524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664233c7-888f-40f6-a13e-c04e993f6339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:31.247622 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664233c7-888f-40f6-a13e-c04e993f6339 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:31.248105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664233c7-888f-40f6-a13e-c04e993f6339 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:31.248617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664233c7-888f-40f6-a13e-c04e993f6339 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:31.264460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664233c7-888f-40f6-a13e-c04e993f6339 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:31.266569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664233c7-888f-40f6-a13e-c04e993f6339 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-664233c7-888f-40f6-a13e-c04e993f6339 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:31.283959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:31.284985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:31.285906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:31.285906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:31.285906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:31.285906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:31.285906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:31.285906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:31.285906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:31.285906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:31.285906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:28:31.285906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:31.288624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 266/548] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:31.295023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190910a-9a2c-409a-a6f2-1ddd3b1dcee7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:31.436107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:31.512251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190910a-9a2c-409a-a6f2-1ddd3b1dcee7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:31.599782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81854234-63ce-499e-baa6-ce4abbb3d5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:31.600855 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81854234-63ce-499e-baa6-ce4abbb3d5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:31.601353 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81854234-63ce-499e-baa6-ce4abbb3d5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:31.601839 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81854234-63ce-499e-baa6-ce4abbb3d5d3 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:31.603288 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81854234-63ce-499e-baa6-ce4abbb3d5d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:31.603699 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81854234-63ce-499e-baa6-ce4abbb3d5d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:31.605410 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81854234-63ce-499e-baa6-ce4abbb3d5d3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:31.639131 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 283/549] 10.222.0.70 () {56 vars in 1135 bytes} [Wed Aug 24 16:28:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:31.647990 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664233c7-888f-40f6-a13e-c04e993f6339 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:31.776386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff330f3-f213-4850-9244-7f552ce1ff82 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:31.776826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff330f3-f213-4850-9244-7f552ce1ff82 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:31.777330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff330f3-f213-4850-9244-7f552ce1ff82 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:31.780089 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 284/550] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:31.786284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66ce94-cdd7-47e9-b94a-1a92b39d7926 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:31.786579 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66ce94-cdd7-47e9-b94a-1a92b39d7926 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:31.786844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66ce94-cdd7-47e9-b94a-1a92b39d7926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:31.826859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4d66ce94-cdd7-47e9-b94a-1a92b39d7926 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:31.829033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4d66ce94-cdd7-47e9-b94a-1a92b39d7926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:31.857039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:31.893422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664233c7-888f-40f6-a13e-c04e993f6339 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:31.989269 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 285/551] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:31.990611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cfcac8-2c8b-4c90-bcd9-8d05da19a1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:31.997991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfcac8-2c8b-4c90-bcd9-8d05da19a1ea None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:31.997991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfcac8-2c8b-4c90-bcd9-8d05da19a1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:31.997991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfcac8-2c8b-4c90-bcd9-8d05da19a1ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:32.010115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0633c2-b00c-4c0e-b36c-3ad89dae597f None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:32.010500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0633c2-b00c-4c0e-b36c-3ad89dae597f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:32.010717 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0633c2-b00c-4c0e-b36c-3ad89dae597f None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:32.018535 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 286/552] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:32.023031 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fabdd-cb96-44e7-85f5-4ec24d426d92 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:32.023427 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fabdd-cb96-44e7-85f5-4ec24d426d92 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:32.023642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fabdd-cb96-44e7-85f5-4ec24d426d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:32.062942 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-183fabdd-cb96-44e7-85f5-4ec24d426d92 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:32.065680 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-183fabdd-cb96-44e7-85f5-4ec24d426d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:32.129486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cfcac8-2c8b-4c90-bcd9-8d05da19a1ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:32.130959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cfcac8-2c8b-4c90-bcd9-8d05da19a1ea None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:32.169931 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:32.195478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 287/553] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:32.233773 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:32.256171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 267/554] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:32.261810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fabdd-cb96-44e7-85f5-4ec24d426d92 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:32.342904 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cfcac8-2c8b-4c90-bcd9-8d05da19a1ea None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:32.448381 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fabdd-cb96-44e7-85f5-4ec24d426d92 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:32.464725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cfcac8-2c8b-4c90-bcd9-8d05da19a1ea None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:32.496819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d354e0-e4ae-4ce0-b889-f8c58179db68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:32.498021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d354e0-e4ae-4ce0-b889-f8c58179db68 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:32.498460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d354e0-e4ae-4ce0-b889-f8c58179db68 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:32.498883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d354e0-e4ae-4ce0-b889-f8c58179db68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:32.506548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce1092-fa6d-402c-a00c-50b8daba935e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:32.507544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce1092-fa6d-402c-a00c-50b8daba935e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:32.508007 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce1092-fa6d-402c-a00c-50b8daba935e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:32.508472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce1092-fa6d-402c-a00c-50b8daba935e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:32.512369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d354e0-e4ae-4ce0-b889-f8c58179db68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:32.514209 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d354e0-e4ae-4ce0-b889-f8c58179db68 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:32.522847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce1092-fa6d-402c-a00c-50b8daba935e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-30ce1092-fa6d-402c-a00c-50b8daba935e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:32.528141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:32.529864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:32.530907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:32.530907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:32.530907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:32.530907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:32.530907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:32.530907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:32.530907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:32.530907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:32.530907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:32.530907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:32.530907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:32.530907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-98d354e0-e4ae-4ce0-b889-f8c58179db68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:32.536384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:32.537788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:32.538653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:32.538653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:32.538653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:32.538653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 288/556] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:28:32.538884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c491ff6-68f4-40d8-a036-7c4afd9ba5b4 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:32.539288 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 268/556] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:32.539388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c491ff6-68f4-40d8-a036-7c4afd9ba5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:32.541507 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c491ff6-68f4-40d8-a036-7c4afd9ba5b4 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:32.541507 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 289/557] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:32.542927 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce1092-fa6d-402c-a00c-50b8daba935e None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:32.545344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c491ff6-68f4-40d8-a036-7c4afd9ba5b4 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:32.600069 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:32.604316 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:32.678733 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce1092-fa6d-402c-a00c-50b8daba935e None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:32.704497 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c491ff6-68f4-40d8-a036-7c4afd9ba5b4 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:32.743278 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40ffaa-bbf0-40c0-a832-403afb3ee418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:32.744241 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40ffaa-bbf0-40c0-a832-403afb3ee418 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:32.744781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40ffaa-bbf0-40c0-a832-403afb3ee418 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:32.745084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40ffaa-bbf0-40c0-a832-403afb3ee418 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:32.745859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40ffaa-bbf0-40c0-a832-403afb3ee418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:32.746232 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40ffaa-bbf0-40c0-a832-403afb3ee418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:32.748709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40ffaa-bbf0-40c0-a832-403afb3ee418 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:32.769257 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51e65a-8887-48ae-9519-8b796a55d20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:32.770211 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51e65a-8887-48ae-9519-8b796a55d20b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:32.770716 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51e65a-8887-48ae-9519-8b796a55d20b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:32.771146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51e65a-8887-48ae-9519-8b796a55d20b None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:32.772505 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51e65a-8887-48ae-9519-8b796a55d20b None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:32.773035 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51e65a-8887-48ae-9519-8b796a55d20b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:32.775622 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51e65a-8887-48ae-9519-8b796a55d20b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:32.776875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 269/558] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:32.779432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cd14d-de1b-4391-846c-c6e283dd7c03 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:32.779887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cd14d-de1b-4391-846c-c6e283dd7c03 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:32.780334 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cd14d-de1b-4391-846c-c6e283dd7c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:32.792443 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 290/559] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:32.793991 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51e65a-8887-48ae-9519-8b796a55d20b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:32.816284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-a85cd14d-de1b-4391-846c-c6e283dd7c03 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:32.818113 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a85cd14d-de1b-4391-846c-c6e283dd7c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:32.935815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 270/560] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:32.937215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cd14d-de1b-4391-846c-c6e283dd7c03 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:32.990944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51e65a-8887-48ae-9519-8b796a55d20b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:32.996441 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:33.008012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:33.083892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab38b06-fbff-4276-bcac-4c00ac929d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:33.086164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab38b06-fbff-4276-bcac-4c00ac929d5c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:33.086998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab38b06-fbff-4276-bcac-4c00ac929d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:33.087794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab38b06-fbff-4276-bcac-4c00ac929d5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:33.119003 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab38b06-fbff-4276-bcac-4c00ac929d5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-9ab38b06-fbff-4276-bcac-4c00ac929d5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:33.129541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:33.131180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:33.132749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:33.132749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:33.132749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:33.132749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:33.132749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:33.132749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:33.132749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:33.132749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:33.132749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:33.132749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:33.132749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:33.132749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:33.132749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 291/561] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:33.134106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab38b06-fbff-4276-bcac-4c00ac929d5c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:33.183519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cd14d-de1b-4391-846c-c6e283dd7c03 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:33.260641 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3710b1a-9932-4ea0-8c03-afc0996b2204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:33.261669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3710b1a-9932-4ea0-8c03-afc0996b2204 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:33.262184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3710b1a-9932-4ea0-8c03-afc0996b2204 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:33.262677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3710b1a-9932-4ea0-8c03-afc0996b2204 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:33.273315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3710b1a-9932-4ea0-8c03-afc0996b2204 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:33.274860 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3710b1a-9932-4ea0-8c03-afc0996b2204 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:33.287511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-d3710b1a-9932-4ea0-8c03-afc0996b2204 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:33.287511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:33.287511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:33.287511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:33.287511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:33.287511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:33.287511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:33.287511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:33.287511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:33.287511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:33.287511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:33.287511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:33.287511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:33.287511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:33.287511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:33.296870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:33.298873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:33.298873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:33.298873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:33.298873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:33.298873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:33.298873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:33.298873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:33.298873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:33.298873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 271/562] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:33.298873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3710b1a-9932-4ea0-8c03-afc0996b2204 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:33.333567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab38b06-fbff-4276-bcac-4c00ac929d5c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:33.385780 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97395c78-0282-482a-b559-0dbc483737de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:33.386886 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97395c78-0282-482a-b559-0dbc483737de None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:33.387377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97395c78-0282-482a-b559-0dbc483737de None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:33.388513 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97395c78-0282-482a-b559-0dbc483737de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:33.404762 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:33.406748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97395c78-0282-482a-b559-0dbc483737de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:33.409268 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97395c78-0282-482a-b559-0dbc483737de None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:33.415070 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3710b1a-9932-4ea0-8c03-afc0996b2204 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-97395c78-0282-482a-b559-0dbc483737de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:33.435916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:33.437394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:33.438859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:33.438859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:33.438859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:33.438859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 292/563] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:33.439281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97395c78-0282-482a-b559-0dbc483737de None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:33.457728 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359a24e-a4ef-4afa-9eb8-a81df78831e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:33.458347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359a24e-a4ef-4afa-9eb8-a81df78831e9 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:33.458625 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359a24e-a4ef-4afa-9eb8-a81df78831e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:33.458895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359a24e-a4ef-4afa-9eb8-a81df78831e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:33.459571 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359a24e-a4ef-4afa-9eb8-a81df78831e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:33.459874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359a24e-a4ef-4afa-9eb8-a81df78831e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:33.461299 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359a24e-a4ef-4afa-9eb8-a81df78831e9 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:33.463301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:33.471477 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 272/564] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:33.472539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7759e2e-2923-482e-8e6d-33d1265d56b5 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:33.472829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7759e2e-2923-482e-8e6d-33d1265d56b5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:33.473103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7759e2e-2923-482e-8e6d-33d1265d56b5 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:33.476480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 273/565] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:33.476480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7759e2e-2923-482e-8e6d-33d1265d56b5 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:33.594448 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97395c78-0282-482a-b559-0dbc483737de None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:33.629874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7759e2e-2923-482e-8e6d-33d1265d56b5 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:33.633484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc0eff-cb34-4264-85a9-047048eb0dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:33.634130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc0eff-cb34-4264-85a9-047048eb0dbb None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:33.634414 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc0eff-cb34-4264-85a9-047048eb0dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:33.634692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc0eff-cb34-4264-85a9-047048eb0dbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:33.646506 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc0eff-cb34-4264-85a9-047048eb0dbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-61dc0eff-cb34-4264-85a9-047048eb0dbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:33.651927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:33.653769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:33.655464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:33.655464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:33.655464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:33.655464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:33.655464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:33.655464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:33.655464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:33.655464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:33.655464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:33.655464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:33.655464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:33.655464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:33.655464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 293/566] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:33.655464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc0eff-cb34-4264-85a9-047048eb0dbb None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:33.687407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7712a0-93c3-449c-ac16-caa91d2a3b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:33.688209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7712a0-93c3-449c-ac16-caa91d2a3b1c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:33.688588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7712a0-93c3-449c-ac16-caa91d2a3b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:33.688929 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7712a0-93c3-449c-ac16-caa91d2a3b1c None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:33.689828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7712a0-93c3-449c-ac16-caa91d2a3b1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:33.690347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7712a0-93c3-449c-ac16-caa91d2a3b1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:33.692156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7712a0-93c3-449c-ac16-caa91d2a3b1c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:33.706743 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 274/567] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:33.707630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7712a0-93c3-449c-ac16-caa91d2a3b1c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:33.735776 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:33.744138 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:33.779859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc0eff-cb34-4264-85a9-047048eb0dbb None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:33.820506 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a6733-623c-4bb5-b0be-38434ce33c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:33.821056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a6733-623c-4bb5-b0be-38434ce33c33 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:33.821337 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a6733-623c-4bb5-b0be-38434ce33c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:33.821638 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a6733-623c-4bb5-b0be-38434ce33c33 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:33.822277 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a6733-623c-4bb5-b0be-38434ce33c33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:33.823029 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a6733-623c-4bb5-b0be-38434ce33c33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:33.824444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a6733-623c-4bb5-b0be-38434ce33c33 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:33.836807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 294/568] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:33.837333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7712a0-93c3-449c-ac16-caa91d2a3b1c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:33.838057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9eda9-c7e1-4549-b960-64e279f05fb0 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:33.838340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9eda9-c7e1-4549-b960-64e279f05fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:33.838614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9eda9-c7e1-4549-b960-64e279f05fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:33.858388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-1ac9eda9-c7e1-4549-b960-64e279f05fb0 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:33.859616 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-1ac9eda9-c7e1-4549-b960-64e279f05fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:33.876989 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62caa20d-c29c-4012-891e-95fa389c7a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:33.877643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62caa20d-c29c-4012-891e-95fa389c7a78 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:33.877926 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62caa20d-c29c-4012-891e-95fa389c7a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:33.878200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62caa20d-c29c-4012-891e-95fa389c7a78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:33.889426 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62caa20d-c29c-4012-891e-95fa389c7a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:33.890924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62caa20d-c29c-4012-891e-95fa389c7a78 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-62caa20d-c29c-4012-891e-95fa389c7a78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:33.909286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:33.911000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:33.912740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:33.912740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:33.912740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:33.912740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 275/569] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:33.912740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62caa20d-c29c-4012-891e-95fa389c7a78 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:33.943015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 295/570] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:33.943686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9eda9-c7e1-4549-b960-64e279f05fb0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:33.998443 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:34.036050 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:34.062075 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62caa20d-c29c-4012-891e-95fa389c7a78 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:34.089801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9eda9-c7e1-4549-b960-64e279f05fb0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:34.107722 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64118464-7bfc-4cc2-8d0e-ffd97ee089b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:34.108342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64118464-7bfc-4cc2-8d0e-ffd97ee089b5 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:34.109071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64118464-7bfc-4cc2-8d0e-ffd97ee089b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:34.109071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64118464-7bfc-4cc2-8d0e-ffd97ee089b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:34.109593 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64118464-7bfc-4cc2-8d0e-ffd97ee089b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:34.109906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64118464-7bfc-4cc2-8d0e-ffd97ee089b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:34.111351 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64118464-7bfc-4cc2-8d0e-ffd97ee089b5 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:34.121406 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 276/571] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:34.126755 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64118464-7bfc-4cc2-8d0e-ffd97ee089b5 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:34.144791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9a116-b010-4628-a516-a0f7800d251b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:34.145416 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9a116-b010-4628-a516-a0f7800d251b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:34.145696 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9a116-b010-4628-a516-a0f7800d251b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:34.145972 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9a116-b010-4628-a516-a0f7800d251b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:34.146721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9a116-b010-4628-a516-a0f7800d251b None admin] RBAC: Adding query filter params (role.id=bb34a44869834493b10a99cc751ab6c2, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=b8ec0b1e5f4d434bb725297f3f867ba7) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:34.147032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9a116-b010-4628-a516-a0f7800d251b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:34.148565 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9a116-b010-4628-a516-a0f7800d251b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:34.155758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 296/572] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:33 2022] GET /identity/v3/role_assignments?user.id=b8ec0b1e5f4d434bb725297f3f867ba7&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=bb34a44869834493b10a99cc751ab6c2 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:34.156672 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9a116-b010-4628-a516-a0f7800d251b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:34.297989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9a116-b010-4628-a516-a0f7800d251b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:34.319741 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64118464-7bfc-4cc2-8d0e-ffd97ee089b5 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:34.333027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:34.343139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692cc1d-afe3-493c-ab98-aba33a562284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:34.343696 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692cc1d-afe3-493c-ab98-aba33a562284 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:34.344060 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692cc1d-afe3-493c-ab98-aba33a562284 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:34.344356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692cc1d-afe3-493c-ab98-aba33a562284 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:34.345057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692cc1d-afe3-493c-ab98-aba33a562284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:34.345367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692cc1d-afe3-493c-ab98-aba33a562284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:34.346762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692cc1d-afe3-493c-ab98-aba33a562284 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:34.358807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:34.361695 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 297/573] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:34.362727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692cc1d-afe3-493c-ab98-aba33a562284 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:34.387726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fffca9-59b3-47cd-a386-048268c89c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:34.389124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fffca9-59b3-47cd-a386-048268c89c9f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:34.389695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fffca9-59b3-47cd-a386-048268c89c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:34.389883 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fffca9-59b3-47cd-a386-048268c89c9f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:34.399281 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fffca9-59b3-47cd-a386-048268c89c9f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:34.401702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fffca9-59b3-47cd-a386-048268c89c9f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-a9fffca9-59b3-47cd-a386-048268c89c9f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:34.416483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:34.418344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:34.420352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:34.420352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:34.420352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:34.420352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:34.420352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:34.420352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:34.420352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:34.420352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:34.420352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:28:34.420352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:34.422941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 277/574] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:34.423768 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fffca9-59b3-47cd-a386-048268c89c9f None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:34.524496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692cc1d-afe3-493c-ab98-aba33a562284 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:34.582735 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fffca9-59b3-47cd-a386-048268c89c9f None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:34.585493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c809d-5894-474e-97a5-bb530f7f5e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:34.586341 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c809d-5894-474e-97a5-bb530f7f5e26 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:34.586882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c809d-5894-474e-97a5-bb530f7f5e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:34.587319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c809d-5894-474e-97a5-bb530f7f5e26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:34.605365 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c809d-5894-474e-97a5-bb530f7f5e26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:34.607542 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c809d-5894-474e-97a5-bb530f7f5e26 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-815c809d-5894-474e-97a5-bb530f7f5e26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:34.628764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:34.630595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:34.633013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:34.633013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:34.633013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:34.633013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 298/575] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:34.638571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c809d-5894-474e-97a5-bb530f7f5e26 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:34.639951 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005434e8-03c3-4c2f-bf5e-c4b40b1baf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:34.641261 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005434e8-03c3-4c2f-bf5e-c4b40b1baf47 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:34.641717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005434e8-03c3-4c2f-bf5e-c4b40b1baf47 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:34.642163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005434e8-03c3-4c2f-bf5e-c4b40b1baf47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:34.664525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005434e8-03c3-4c2f-bf5e-c4b40b1baf47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-005434e8-03c3-4c2f-bf5e-c4b40b1baf47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:34.673401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:34.674438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:34.675267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:34.675267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:34.675267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:34.675267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:34.675267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:34.675267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:34.675267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:34.675267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:34.675267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:34.675267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:34.675267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:34.675267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:34.681930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 278/576] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:34.681930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005434e8-03c3-4c2f-bf5e-c4b40b1baf47 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:34.716520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:34.767976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:34.807345 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c809d-5894-474e-97a5-bb530f7f5e26 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:34.837149 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005434e8-03c3-4c2f-bf5e-c4b40b1baf47 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:34.870400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19272ae2-be26-48e9-8cbd-f1b8986b227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:34.871439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19272ae2-be26-48e9-8cbd-f1b8986b227e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:34.871966 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19272ae2-be26-48e9-8cbd-f1b8986b227e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:34.873072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19272ae2-be26-48e9-8cbd-f1b8986b227e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:34.877159 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f74f15-39b3-4f61-a5c9-5adb5ea1bcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:34.877778 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f74f15-39b3-4f61-a5c9-5adb5ea1bcd9 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:34.878507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f74f15-39b3-4f61-a5c9-5adb5ea1bcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:34.878507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f74f15-39b3-4f61-a5c9-5adb5ea1bcd9 None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:34.879114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f74f15-39b3-4f61-a5c9-5adb5ea1bcd9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:34.879342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f74f15-39b3-4f61-a5c9-5adb5ea1bcd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:34.880829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f74f15-39b3-4f61-a5c9-5adb5ea1bcd9 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:34.888496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19272ae2-be26-48e9-8cbd-f1b8986b227e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:34.890847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19272ae2-be26-48e9-8cbd-f1b8986b227e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:34.905619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 279/577] 10.222.0.70 () {56 vars in 1137 bytes} [Wed Aug 24 16:28:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:34.905619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f74f15-39b3-4f61-a5c9-5adb5ea1bcd9 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-19272ae2-be26-48e9-8cbd-f1b8986b227e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:34.910782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:34.911767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:34.912768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:34.912768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:34.912768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:34.912768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:34.912768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:34.912768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:34.912768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:34.912768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:34.912768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:28:34.912768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:34.913317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 299/578] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:34.914477 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19272ae2-be26-48e9-8cbd-f1b8986b227e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:35.065656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f74f15-39b3-4f61-a5c9-5adb5ea1bcd9 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:35.076089 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:35.078237 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:35.080647 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19272ae2-be26-48e9-8cbd-f1b8986b227e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:35.109850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9275c1-abc8-45fe-99df-f5446cee6acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:35.110485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9275c1-abc8-45fe-99df-f5446cee6acd None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:35.110767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9275c1-abc8-45fe-99df-f5446cee6acd None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:35.111039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9275c1-abc8-45fe-99df-f5446cee6acd None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:35.111697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9275c1-abc8-45fe-99df-f5446cee6acd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:35.112003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9275c1-abc8-45fe-99df-f5446cee6acd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:35.113717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9275c1-abc8-45fe-99df-f5446cee6acd None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:35.126408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd0d54-384e-4340-a8b6-040f891aec82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:35.126408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd0d54-384e-4340-a8b6-040f891aec82 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:35.126408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd0d54-384e-4340-a8b6-040f891aec82 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:35.126408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd0d54-384e-4340-a8b6-040f891aec82 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:35.126852 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd0d54-384e-4340-a8b6-040f891aec82 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:35.126852 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd0d54-384e-4340-a8b6-040f891aec82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:35.127356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd0d54-384e-4340-a8b6-040f891aec82 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:35.127990 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 280/579] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:35.129049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9275c1-abc8-45fe-99df-f5446cee6acd None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:35.141823 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 300/580] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:35.147213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd0d54-384e-4340-a8b6-040f891aec82 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:35.257216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9275c1-abc8-45fe-99df-f5446cee6acd None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:35.310648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd0d54-384e-4340-a8b6-040f891aec82 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:35.328018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcb10a-e770-4718-afe4-7bde20dca965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:35.328913 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcb10a-e770-4718-afe4-7bde20dca965 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:35.329237 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcb10a-e770-4718-afe4-7bde20dca965 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:35.329681 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcb10a-e770-4718-afe4-7bde20dca965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:35.342661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bcb10a-e770-4718-afe4-7bde20dca965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:35.344177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bcb10a-e770-4718-afe4-7bde20dca965 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:35.358335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e42db-d488-445c-9037-b6c767ef966d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:35.359139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e42db-d488-445c-9037-b6c767ef966d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:35.359561 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e42db-d488-445c-9037-b6c767ef966d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:35.359964 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e42db-d488-445c-9037-b6c767ef966d None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:35.360932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e42db-d488-445c-9037-b6c767ef966d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:35.361379 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e42db-d488-445c-9037-b6c767ef966d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-45bcb10a-e770-4718-afe4-7bde20dca965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:35.361807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:35.363493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:35.365400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:35.365400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:35.365400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:35.365748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e42db-d488-445c-9037-b6c767ef966d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:35.368149 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 281/581] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:35.369293 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcb10a-e770-4718-afe4-7bde20dca965 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:35.386114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 301/582] 10.222.0.70 () {56 vars in 1137 bytes} [Wed Aug 24 16:28:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:35.386920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e42db-d488-445c-9037-b6c767ef966d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:35.398943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:35.421959 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:35.531169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e42db-d488-445c-9037-b6c767ef966d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:35.532659 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcb10a-e770-4718-afe4-7bde20dca965 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:35.580803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d65a6-ebc1-43ae-b478-16cbfc8b9c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:35.581495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d65a6-ebc1-43ae-b478-16cbfc8b9c58 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:35.581961 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d65a6-ebc1-43ae-b478-16cbfc8b9c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:35.582125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d65a6-ebc1-43ae-b478-16cbfc8b9c58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:35.598872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d65a6-ebc1-43ae-b478-16cbfc8b9c58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:35.600001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5203a85-4e26-4a21-9803-2ee2c20ebccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:35.600690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5203a85-4e26-4a21-9803-2ee2c20ebccb None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:35.600971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5203a85-4e26-4a21-9803-2ee2c20ebccb None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:35.601247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5203a85-4e26-4a21-9803-2ee2c20ebccb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:35.601625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d65a6-ebc1-43ae-b478-16cbfc8b9c58 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:35.612602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5203a85-4e26-4a21-9803-2ee2c20ebccb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:35.614764 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5203a85-4e26-4a21-9803-2ee2c20ebccb None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-a5203a85-4e26-4a21-9803-2ee2c20ebccb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:35.630322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:35.637528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:35.639277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:35.639277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:35.639277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:35.639277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:35.639277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:35.639277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:35.639277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:35.639277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:35.639277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:28:35.639277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-047d65a6-ebc1-43ae-b478-16cbfc8b9c58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:35.645206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:35.648564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:35.650555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:35.650555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:35.650555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:35.650938 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 282/583] 10.222.0.70 () {56 vars in 1093 bytes} [Wed Aug 24 16:28:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:35.650938 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5203a85-4e26-4a21-9803-2ee2c20ebccb None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:35.651209 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 302/584] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:35.651209 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d65a6-ebc1-43ae-b478-16cbfc8b9c58 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:35.737870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:35.762403 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:35.782043 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5203a85-4e26-4a21-9803-2ee2c20ebccb None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:35.807582 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d65a6-ebc1-43ae-b478-16cbfc8b9c58 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:35.826429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a511cd-1006-4dbc-b8a0-bd322649c51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:35.827062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a511cd-1006-4dbc-b8a0-bd322649c51a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:35.827340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a511cd-1006-4dbc-b8a0-bd322649c51a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:35.827612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a511cd-1006-4dbc-b8a0-bd322649c51a None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:35.828893 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a511cd-1006-4dbc-b8a0-bd322649c51a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:35.829217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a511cd-1006-4dbc-b8a0-bd322649c51a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:35.830635 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a511cd-1006-4dbc-b8a0-bd322649c51a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:35.853951 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 283/585] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:35.853951 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a511cd-1006-4dbc-b8a0-bd322649c51a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:35.880525 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80abaf-104f-4a35-a66f-81684a3400ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:35.881738 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80abaf-104f-4a35-a66f-81684a3400ab None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:35.882204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80abaf-104f-4a35-a66f-81684a3400ab None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:35.882683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80abaf-104f-4a35-a66f-81684a3400ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:35.903943 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80abaf-104f-4a35-a66f-81684a3400ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:35.906741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80abaf-104f-4a35-a66f-81684a3400ab None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-da80abaf-104f-4a35-a66f-81684a3400ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:35.933367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:35.934534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:35.935677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:35.935677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:35.935677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:35.942946 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 303/586] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:35.942946 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80abaf-104f-4a35-a66f-81684a3400ab None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:36.026567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a511cd-1006-4dbc-b8a0-bd322649c51a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:36.076886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e66fa0-0296-4001-8e45-d09a0c1bd0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:36.077501 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e66fa0-0296-4001-8e45-d09a0c1bd0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:36.077775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e66fa0-0296-4001-8e45-d09a0c1bd0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:36.078130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e66fa0-0296-4001-8e45-d09a0c1bd0d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:36.078850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e66fa0-0296-4001-8e45-d09a0c1bd0d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:36.079156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e66fa0-0296-4001-8e45-d09a0c1bd0d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:36.080719 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e66fa0-0296-4001-8e45-d09a0c1bd0d4 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:36.089563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:36.102332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 284/587] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:36.102332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e66fa0-0296-4001-8e45-d09a0c1bd0d4 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:36.139522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80abaf-104f-4a35-a66f-81684a3400ab None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:36.174890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:36.192202 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca21e2-feb9-45ba-b758-41cd09ffeec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:36.193191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca21e2-feb9-45ba-b758-41cd09ffeec0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:36.193622 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca21e2-feb9-45ba-b758-41cd09ffeec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:36.194079 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca21e2-feb9-45ba-b758-41cd09ffeec0 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:36.195431 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca21e2-feb9-45ba-b758-41cd09ffeec0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:36.195942 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca21e2-feb9-45ba-b758-41cd09ffeec0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:36.198451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca21e2-feb9-45ba-b758-41cd09ffeec0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:36.227573 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 304/588] 10.222.0.70 () {56 vars in 1133 bytes} [Wed Aug 24 16:28:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:36.228683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca21e2-feb9-45ba-b758-41cd09ffeec0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:36.238183 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e66fa0-0296-4001-8e45-d09a0c1bd0d4 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:36.277930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdc30d-69b9-498e-9e7b-953bfd5c6ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:36.279131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdc30d-69b9-498e-9e7b-953bfd5c6ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:36.279400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdc30d-69b9-498e-9e7b-953bfd5c6ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:36.279666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdc30d-69b9-498e-9e7b-953bfd5c6ad6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:36.287707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdc30d-69b9-498e-9e7b-953bfd5c6ad6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:36.289236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdc30d-69b9-498e-9e7b-953bfd5c6ad6 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:36.302181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-dffdc30d-69b9-498e-9e7b-953bfd5c6ad6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:36.302181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:36.302181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:36.302181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:36.302181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:36.302181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:36.302181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:36.302181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:36.302181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:36.302181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:36.302181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:36.302181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:36.302181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:36.302181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:36.302181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:36.303263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:36.304779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:36.304779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:36.304779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:36.304779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:36.304779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:36.304779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:36.304779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:36.304779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:36.304779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 285/589] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:36.304779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdc30d-69b9-498e-9e7b-953bfd5c6ad6 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:36.374774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:36.382438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca21e2-feb9-45ba-b758-41cd09ffeec0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:36.435304 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f0f6d-a79a-40e7-9eb9-853d2af114e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:36.436097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f0f6d-a79a-40e7-9eb9-853d2af114e8 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:36.436585 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f0f6d-a79a-40e7-9eb9-853d2af114e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:36.437049 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f0f6d-a79a-40e7-9eb9-853d2af114e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:36.438153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f0f6d-a79a-40e7-9eb9-853d2af114e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:36.438656 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f0f6d-a79a-40e7-9eb9-853d2af114e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:36.441114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f0f6d-a79a-40e7-9eb9-853d2af114e8 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:36.441892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdc30d-69b9-498e-9e7b-953bfd5c6ad6 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:36.460425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 305/590] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:36.461902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07d943-4dfa-440b-b251-dde5b570c1b4 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:36.462366 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07d943-4dfa-440b-b251-dde5b570c1b4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:36.462766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07d943-4dfa-440b-b251-dde5b570c1b4 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:36.468984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 306/591] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:36.468984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07d943-4dfa-440b-b251-dde5b570c1b4 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:36.499150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb037cd-e892-48e9-847a-fa6204bd64e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:36.499845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb037cd-e892-48e9-847a-fa6204bd64e2 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:36.500129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb037cd-e892-48e9-847a-fa6204bd64e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:36.500428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb037cd-e892-48e9-847a-fa6204bd64e2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:36.509655 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb037cd-e892-48e9-847a-fa6204bd64e2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:36.511639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb037cd-e892-48e9-847a-fa6204bd64e2 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:36.521204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-8eb037cd-e892-48e9-847a-fa6204bd64e2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:36.525358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:36.533306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:36.535177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:36.535177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:36.535177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:36.535177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:36.535177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:36.535177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:36.535177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:36.535177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:36.535177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:28:36.535177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:36.535177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 286/592] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:36.535177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb037cd-e892-48e9-847a-fa6204bd64e2 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:36.606537 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07d943-4dfa-440b-b251-dde5b570c1b4 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:36.651811 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63988f-8ec4-4ef9-8787-480bddc0a20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:36.652651 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63988f-8ec4-4ef9-8787-480bddc0a20f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:36.653023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63988f-8ec4-4ef9-8787-480bddc0a20f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:36.654015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63988f-8ec4-4ef9-8787-480bddc0a20f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:36.664659 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb037cd-e892-48e9-847a-fa6204bd64e2 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:36.671678 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63988f-8ec4-4ef9-8787-480bddc0a20f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:36.674243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63988f-8ec4-4ef9-8787-480bddc0a20f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:36.704507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-da63988f-8ec4-4ef9-8787-480bddc0a20f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:36.704694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:36.706253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:36.707584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:36.707584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:36.707584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:36.707584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 307/593] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:36.707584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63988f-8ec4-4ef9-8787-480bddc0a20f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:36.712820 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dd9ce-36ad-4748-96f7-3a0ff1f65857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:36.713491 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dd9ce-36ad-4748-96f7-3a0ff1f65857 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:36.713775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dd9ce-36ad-4748-96f7-3a0ff1f65857 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:36.714046 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dd9ce-36ad-4748-96f7-3a0ff1f65857 None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:36.714751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263dd9ce-36ad-4748-96f7-3a0ff1f65857 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:36.715055 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263dd9ce-36ad-4748-96f7-3a0ff1f65857 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:36.717099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263dd9ce-36ad-4748-96f7-3a0ff1f65857 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:36.730769 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 287/594] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:36.732160 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb05414-0784-4f92-b7e0-959137983a15 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:36.732309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb05414-0784-4f92-b7e0-959137983a15 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:36.733163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb05414-0784-4f92-b7e0-959137983a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:36.752309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-aeb05414-0784-4f92-b7e0-959137983a15 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:36.753637 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-aeb05414-0784-4f92-b7e0-959137983a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:36.818782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:36.826861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63988f-8ec4-4ef9-8787-480bddc0a20f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:36.853107 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 288/595] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:36.853993 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb05414-0784-4f92-b7e0-959137983a15 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:36.875680 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee309ea5-8d2e-4667-83dd-3c57c7114ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:36.876755 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee309ea5-8d2e-4667-83dd-3c57c7114ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:36.877298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee309ea5-8d2e-4667-83dd-3c57c7114ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:36.878594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee309ea5-8d2e-4667-83dd-3c57c7114ec3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:36.890742 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee309ea5-8d2e-4667-83dd-3c57c7114ec3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:36.893913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee309ea5-8d2e-4667-83dd-3c57c7114ec3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:36.909587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-ee309ea5-8d2e-4667-83dd-3c57c7114ec3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:36.909587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:36.909587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:36.909587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:36.909587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:36.909587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:36.909587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:36.909587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:36.909587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:36.909587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:36.909587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:36.909587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:36.909587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:36.909587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:36.909587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:36.910546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:36.911458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:36.911458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:36.911458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:36.911458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:36.911458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:36.911458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:36.911458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:36.911458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:36.911458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 308/596] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:36.911943 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee309ea5-8d2e-4667-83dd-3c57c7114ec3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:37.003344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb05414-0784-4f92-b7e0-959137983a15 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:37.021883 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:37.034146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee309ea5-8d2e-4667-83dd-3c57c7114ec3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:37.072063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130beae5-a909-4bd1-bde1-ae53305bd334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:37.073268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130beae5-a909-4bd1-bde1-ae53305bd334 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:37.073838 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130beae5-a909-4bd1-bde1-ae53305bd334 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:37.074424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130beae5-a909-4bd1-bde1-ae53305bd334 None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:37.075976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130beae5-a909-4bd1-bde1-ae53305bd334 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:37.076558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130beae5-a909-4bd1-bde1-ae53305bd334 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:37.079458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130beae5-a909-4bd1-bde1-ae53305bd334 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:37.100985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7eeaee-8060-4264-b8ba-fb714a4dc52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:37.102027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7eeaee-8060-4264-b8ba-fb714a4dc52e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:37.102524 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7eeaee-8060-4264-b8ba-fb714a4dc52e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:37.103013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7eeaee-8060-4264-b8ba-fb714a4dc52e None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:37.103958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7eeaee-8060-4264-b8ba-fb714a4dc52e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:37.104455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7eeaee-8060-4264-b8ba-fb714a4dc52e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:37.106991 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7eeaee-8060-4264-b8ba-fb714a4dc52e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:37.126016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 289/597] 10.222.0.70 () {56 vars in 1139 bytes} [Wed Aug 24 16:28:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:37.126016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130beae5-a909-4bd1-bde1-ae53305bd334 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:37.140816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 309/598] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:37.142523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7eeaee-8060-4264-b8ba-fb714a4dc52e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:37.173331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:37.267746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130beae5-a909-4bd1-bde1-ae53305bd334 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:37.311175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a52a70-e6e7-4e57-a8d0-7d52c39294e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:37.311446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7eeaee-8060-4264-b8ba-fb714a4dc52e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:37.311855 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a52a70-e6e7-4e57-a8d0-7d52c39294e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:37.312139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a52a70-e6e7-4e57-a8d0-7d52c39294e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:37.312448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a52a70-e6e7-4e57-a8d0-7d52c39294e8 None admin] PATH_INFO: `/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/f08140bcbd1e43dda77d4387666f9085/roles/bb34a44869834493b10a99cc751ab6c2` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:37.329645 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a52a70-e6e7-4e57-a8d0-7d52c39294e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc920dce30734ac7a69dc039d8c74ce1, user_id=f08140bcbd1e43dda77d4387666f9085, role_id=bb34a44869834493b10a99cc751ab6c2)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:37.331109 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a52a70-e6e7-4e57-a8d0-7d52c39294e8 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:37.355319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ca02f-a9d7-46c6-8a64-7cf0d6aefbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:37.356103 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ca02f-a9d7-46c6-8a64-7cf0d6aefbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:37.357055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ca02f-a9d7-46c6-8a64-7cf0d6aefbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:37.357412 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ca02f-a9d7-46c6-8a64-7cf0d6aefbf9 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:37.358201 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ca02f-a9d7-46c6-8a64-7cf0d6aefbf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:37.358591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ca02f-a9d7-46c6-8a64-7cf0d6aefbf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:37.360188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ca02f-a9d7-46c6-8a64-7cf0d6aefbf9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:37.374358 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 310/599] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:37.374953 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ca02f-a9d7-46c6-8a64-7cf0d6aefbf9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:37.411951 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 290/600] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:28:37 2022] PUT /identity/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/f08140bcbd1e43dda77d4387666f9085/roles/bb34a44869834493b10a99cc751ab6c2 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:28:37.413535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a52a70-e6e7-4e57-a8d0-7d52c39294e8 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:37.443901 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:37.478849 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:37.524476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ca02f-a9d7-46c6-8a64-7cf0d6aefbf9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:37.580244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52a97d-0655-4c12-9427-4452902feb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:37.582805 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52a97d-0655-4c12-9427-4452902feb83 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:37.582805 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52a97d-0655-4c12-9427-4452902feb83 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:37.584754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52a97d-0655-4c12-9427-4452902feb83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:37.590336 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a52a70-e6e7-4e57-a8d0-7d52c39294e8 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:37.609784 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52a97d-0655-4c12-9427-4452902feb83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:37.612583 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52a97d-0655-4c12-9427-4452902feb83 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-1a52a97d-0655-4c12-9427-4452902feb83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:37.633732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:37.634958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:37.636171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:37.636171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:37.636171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:37.642662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 311/601] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:37.642662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52a97d-0655-4c12-9427-4452902feb83 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:37.643848 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338fe8d-e3bf-4213-b88e-513e39e83e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:37.644942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338fe8d-e3bf-4213-b88e-513e39e83e71 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:37.645399 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338fe8d-e3bf-4213-b88e-513e39e83e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:37.645873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338fe8d-e3bf-4213-b88e-513e39e83e71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:37.658913 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338fe8d-e3bf-4213-b88e-513e39e83e71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:37.662564 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338fe8d-e3bf-4213-b88e-513e39e83e71 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:37.676700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-8338fe8d-e3bf-4213-b88e-513e39e83e71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:37.676700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:37.676700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:37.676700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:37.676700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:37.676700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:37.676700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:37.676700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:37.676700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:37.676700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:37.676700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:37.676700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:37.676700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:37.676700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:37.676700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:37.678508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:37.680301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:37.680301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:37.680301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:37.680301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:37.680301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:37.680301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:37.680301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:37.680301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:37.680301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 291/602] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:37.680301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338fe8d-e3bf-4213-b88e-513e39e83e71 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:37.771469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:37.773330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52a97d-0655-4c12-9427-4452902feb83 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:37.797190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338fe8d-e3bf-4213-b88e-513e39e83e71 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:37.803514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:37.815983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fa80c-677f-4e73-bb5f-d53ca707cdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:37.816751 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fa80c-677f-4e73-bb5f-d53ca707cdef None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:37.817161 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fa80c-677f-4e73-bb5f-d53ca707cdef None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:37.817628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fa80c-677f-4e73-bb5f-d53ca707cdef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:37.835227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fa80c-677f-4e73-bb5f-d53ca707cdef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-3b8fa80c-677f-4e73-bb5f-d53ca707cdef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:37.840475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:37.842163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:37.843015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:37.843015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:37.843015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:37.843015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:37.843015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:37.843015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:37.843015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:37.843015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:37.843015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:37.843015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:37.843015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:37.843015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:37.843015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 312/603] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:37.843743 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fa80c-677f-4e73-bb5f-d53ca707cdef None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:37.845173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdb47b-96a9-4bed-a366-4738afdf9b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:37.845836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdb47b-96a9-4bed-a366-4738afdf9b8e None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:37.846113 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdb47b-96a9-4bed-a366-4738afdf9b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:37.846382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdb47b-96a9-4bed-a366-4738afdf9b8e None admin] PATH_INFO: `/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/56418ff737504181a59714aad9f1c375/roles/bb34a44869834493b10a99cc751ab6c2` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:37.864671 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacdb47b-96a9-4bed-a366-4738afdf9b8e None admin] RBAC: Authorizing `identity:create_grant(project_id=cc920dce30734ac7a69dc039d8c74ce1, user_id=56418ff737504181a59714aad9f1c375, role_id=bb34a44869834493b10a99cc751ab6c2)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:37.866808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacdb47b-96a9-4bed-a366-4738afdf9b8e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:37.937398 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 292/604] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:28:37 2022] PUT /identity/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/56418ff737504181a59714aad9f1c375/roles/bb34a44869834493b10a99cc751ab6c2 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:28:37.938253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdb47b-96a9-4bed-a366-4738afdf9b8e None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:37.972357 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fa80c-677f-4e73-bb5f-d53ca707cdef None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:38.017306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc01998-76e0-4f54-a123-8f95e09c2985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:38.019039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc01998-76e0-4f54-a123-8f95e09c2985 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:38.019567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc01998-76e0-4f54-a123-8f95e09c2985 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:38.020055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc01998-76e0-4f54-a123-8f95e09c2985 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:38.021050 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc01998-76e0-4f54-a123-8f95e09c2985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:38.021639 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc01998-76e0-4f54-a123-8f95e09c2985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:38.024141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc01998-76e0-4f54-a123-8f95e09c2985 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:38.047567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 313/605] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:38.047567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc01998-76e0-4f54-a123-8f95e09c2985 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:38.098416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdb47b-96a9-4bed-a366-4738afdf9b8e None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:38.140326 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:38.143361 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:38.158073 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e8b5e-ab3f-473c-b3f4-2e50e28a7d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:38.159268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e8b5e-ab3f-473c-b3f4-2e50e28a7d0a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:38.160089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e8b5e-ab3f-473c-b3f4-2e50e28a7d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:38.161605 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e8b5e-ab3f-473c-b3f4-2e50e28a7d0a None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:38.162017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e8b5e-ab3f-473c-b3f4-2e50e28a7d0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:38.162460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e8b5e-ab3f-473c-b3f4-2e50e28a7d0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:38.166103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e8b5e-ab3f-473c-b3f4-2e50e28a7d0a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:38.191892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 293/606] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:38.193363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e8b5e-ab3f-473c-b3f4-2e50e28a7d0a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:38.237975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc01998-76e0-4f54-a123-8f95e09c2985 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:38.312800 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf1b7c-2092-4e80-8531-1c5602f55a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:38.315255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf1b7c-2092-4e80-8531-1c5602f55a7d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:38.315756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf1b7c-2092-4e80-8531-1c5602f55a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:38.316664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf1b7c-2092-4e80-8531-1c5602f55a7d None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:38.318388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf1b7c-2092-4e80-8531-1c5602f55a7d None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:38.319632 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf1b7c-2092-4e80-8531-1c5602f55a7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:38.324121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf1b7c-2092-4e80-8531-1c5602f55a7d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:38.349013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 314/607] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:38.351337 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf1b7c-2092-4e80-8531-1c5602f55a7d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:38.378793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e8b5e-ab3f-473c-b3f4-2e50e28a7d0a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:38.448211 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebc58d-e14f-4342-a1a2-64c42fda0ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:38.449130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebc58d-e14f-4342-a1a2-64c42fda0ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:38.449509 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebc58d-e14f-4342-a1a2-64c42fda0ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:38.449936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebc58d-e14f-4342-a1a2-64c42fda0ba1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:38.458958 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebc58d-e14f-4342-a1a2-64c42fda0ba1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:38.460645 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebc58d-e14f-4342-a1a2-64c42fda0ba1 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:38.478084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-5cebc58d-e14f-4342-a1a2-64c42fda0ba1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:38.478084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:38.478084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:38.478084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:38.478084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:38.478084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:38.478084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:38.478084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:38.478084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:38.478084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:38.478084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:38.478084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:38.478084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:38.478084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:38.478084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:38.479115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:38.480365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:38.480365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:38.480365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:38.480365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:38.480365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:38.480365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:38.480365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:38.480365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:38.480365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 294/608] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:38.482507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebc58d-e14f-4342-a1a2-64c42fda0ba1 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:38.543685 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf1b7c-2092-4e80-8531-1c5602f55a7d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:38.557905 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:38.560053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:38.594115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672a7af-0d6c-452e-ba8f-b75d90575357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:38.594686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672a7af-0d6c-452e-ba8f-b75d90575357 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:38.595207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672a7af-0d6c-452e-ba8f-b75d90575357 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:38.595734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672a7af-0d6c-452e-ba8f-b75d90575357 None admin] PATH_INFO: `/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/e333fca7790244f6a4ebceabb55faba4/roles/bb34a44869834493b10a99cc751ab6c2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:38.621229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672a7af-0d6c-452e-ba8f-b75d90575357 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc920dce30734ac7a69dc039d8c74ce1, user_id=e333fca7790244f6a4ebceabb55faba4, role_id=bb34a44869834493b10a99cc751ab6c2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:38.623369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672a7af-0d6c-452e-ba8f-b75d90575357 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:38.663242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebc58d-e14f-4342-a1a2-64c42fda0ba1 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:38.718275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25d31c-d148-4971-a7be-a015d3c061b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:38.719391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25d31c-d148-4971-a7be-a015d3c061b2 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:38.719875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25d31c-d148-4971-a7be-a015d3c061b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:38.720346 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25d31c-d148-4971-a7be-a015d3c061b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:38.742148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25d31c-d148-4971-a7be-a015d3c061b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:38.745019 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25d31c-d148-4971-a7be-a015d3c061b2 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:38.769683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 315/609] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:28:38 2022] PUT /identity/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/e333fca7790244f6a4ebceabb55faba4/roles/bb34a44869834493b10a99cc751ab6c2 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:28:38.771479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672a7af-0d6c-452e-ba8f-b75d90575357 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-cd25d31c-d148-4971-a7be-a015d3c061b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:38.787981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:38.790175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:38.791999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:38.791999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:38.791999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:38.798042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 295/610] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:38.803646 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25d31c-d148-4971-a7be-a015d3c061b2 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:39.040517 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:39.048440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25d31c-d148-4971-a7be-a015d3c061b2 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:39.065857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672a7af-0d6c-452e-ba8f-b75d90575357 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:39.097486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05791cf0-a78f-4929-8761-c21d6298919b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:39.098407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05791cf0-a78f-4929-8761-c21d6298919b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:39.099230 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05791cf0-a78f-4929-8761-c21d6298919b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:39.099519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05791cf0-a78f-4929-8761-c21d6298919b None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:39.100824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05791cf0-a78f-4929-8761-c21d6298919b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:39.101219 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05791cf0-a78f-4929-8761-c21d6298919b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:39.103594 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05791cf0-a78f-4929-8761-c21d6298919b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:39.115555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7699e-ebad-4ad8-a0b1-63d024527f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:39.116255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7699e-ebad-4ad8-a0b1-63d024527f61 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:39.116592 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7699e-ebad-4ad8-a0b1-63d024527f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:39.116971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7699e-ebad-4ad8-a0b1-63d024527f61 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:39.117737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c7699e-ebad-4ad8-a0b1-63d024527f61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:39.118046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c7699e-ebad-4ad8-a0b1-63d024527f61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:39.120541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c7699e-ebad-4ad8-a0b1-63d024527f61 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:39.122386 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 296/611] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:39.134426 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05791cf0-a78f-4929-8761-c21d6298919b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:39.142784 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 316/612] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:39.158519 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7699e-ebad-4ad8-a0b1-63d024527f61 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:39.199469 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:39.391601 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05791cf0-a78f-4929-8761-c21d6298919b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:39.418784 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7699e-ebad-4ad8-a0b1-63d024527f61 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:39.477754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42af98-7917-4060-b477-eac44e742f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:39.478416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42af98-7917-4060-b477-eac44e742f09 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:39.478700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42af98-7917-4060-b477-eac44e742f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:39.478978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42af98-7917-4060-b477-eac44e742f09 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:39.490660 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42af98-7917-4060-b477-eac44e742f09 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:39.492143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42af98-7917-4060-b477-eac44e742f09 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-1d42af98-7917-4060-b477-eac44e742f09 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:39.512468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:39.513551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:39.514483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:39.514483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:39.514483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:39.514483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:39.514483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:39.514483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:39.514483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:39.514483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:39.514483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:28:39.514483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:39.514483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 297/613] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:28:39.528438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42af98-7917-4060-b477-eac44e742f09 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:39.536438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad60a4-349f-46c5-911a-2019518ea0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:39.536438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad60a4-349f-46c5-911a-2019518ea0ac None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:39.536438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad60a4-349f-46c5-911a-2019518ea0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:39.536438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad60a4-349f-46c5-911a-2019518ea0ac None admin] PATH_INFO: `/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/942b54c39ae5403c8ae3bcca1c31bf1a/roles/bb34a44869834493b10a99cc751ab6c2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:39.563887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:39.568376 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad60a4-349f-46c5-911a-2019518ea0ac None admin] RBAC: Authorizing `identity:create_grant(project_id=cc920dce30734ac7a69dc039d8c74ce1, user_id=942b54c39ae5403c8ae3bcca1c31bf1a, role_id=bb34a44869834493b10a99cc751ab6c2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:39.570273 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad60a4-349f-46c5-911a-2019518ea0ac None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:39.680120 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42af98-7917-4060-b477-eac44e742f09 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:39.687881 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 317/614] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:28:39 2022] PUT /identity/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/942b54c39ae5403c8ae3bcca1c31bf1a/roles/bb34a44869834493b10a99cc751ab6c2 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:28:39.754038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f396ac-bf09-4d1d-87a5-184bc28cf3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:39.755095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f396ac-bf09-4d1d-87a5-184bc28cf3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:39.755593 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f396ac-bf09-4d1d-87a5-184bc28cf3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:39.756105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f396ac-bf09-4d1d-87a5-184bc28cf3f5 None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:39.757176 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f396ac-bf09-4d1d-87a5-184bc28cf3f5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:39.757705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f396ac-bf09-4d1d-87a5-184bc28cf3f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:39.759435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f396ac-bf09-4d1d-87a5-184bc28cf3f5 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:39.795228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 298/615] 10.222.0.70 () {56 vars in 1135 bytes} [Wed Aug 24 16:28:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:39.801347 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad60a4-349f-46c5-911a-2019518ea0ac None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:39.875362 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4178e7-1f4c-4021-b363-32f852c3c7fa None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:39.876018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4178e7-1f4c-4021-b363-32f852c3c7fa None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:39.876543 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4178e7-1f4c-4021-b363-32f852c3c7fa None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:39.878498 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 299/616] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:39.888086 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e3e71-4e30-4a90-9664-1431ff1b4139 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:39.888458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e3e71-4e30-4a90-9664-1431ff1b4139 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:39.888677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e3e71-4e30-4a90-9664-1431ff1b4139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:39.921972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-382e3e71-4e30-4a90-9664-1431ff1b4139 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:39.923218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-382e3e71-4e30-4a90-9664-1431ff1b4139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:40.011732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:40.075685 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad60a4-349f-46c5-911a-2019518ea0ac None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:40.092434 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 300/617] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 16:28:40.194948 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abed2f-0703-4d56-bdb2-e6cc710ea855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:40.195943 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abed2f-0703-4d56-bdb2-e6cc710ea855 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:40.196436 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abed2f-0703-4d56-bdb2-e6cc710ea855 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:40.196865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abed2f-0703-4d56-bdb2-e6cc710ea855 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:40.221778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abed2f-0703-4d56-bdb2-e6cc710ea855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:40.223521 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abed2f-0703-4d56-bdb2-e6cc710ea855 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-09abed2f-0703-4d56-bdb2-e6cc710ea855 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:40.255523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:40.261088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:40.262144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:40.262144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:40.262144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:40.264889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 318/618] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 464 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:40.271131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e3e71-4e30-4a90-9664-1431ff1b4139 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:40.289823 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:40.385996 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d612602-a278-4414-94d5-19e46cbf87c2 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:40.386910 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d612602-a278-4414-94d5-19e46cbf87c2 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:40.388065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d612602-a278-4414-94d5-19e46cbf87c2 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:40.391928 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 319/619] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:28:40.396630 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dd1ab-2296-46e7-a338-49de4fe827ba None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:40.401646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dd1ab-2296-46e7-a338-49de4fe827ba None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:40.402426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dd1ab-2296-46e7-a338-49de4fe827ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:40.422558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e3e71-4e30-4a90-9664-1431ff1b4139 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:40.467073 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e1809-9764-4ef4-88ef-f8e12e352e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:40.467725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e1809-9764-4ef4-88ef-f8e12e352e1f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:40.468006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e1809-9764-4ef4-88ef-f8e12e352e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:40.468290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e1809-9764-4ef4-88ef-f8e12e352e1f None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:40.468941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e1809-9764-4ef4-88ef-f8e12e352e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:40.469290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e1809-9764-4ef4-88ef-f8e12e352e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:40.474582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e1809-9764-4ef4-88ef-f8e12e352e1f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:40.482773 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-eb8dd1ab-2296-46e7-a338-49de4fe827ba None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:40.484090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-eb8dd1ab-2296-46e7-a338-49de4fe827ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:40.486405 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 301/620] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:40.500429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e1809-9764-4ef4-88ef-f8e12e352e1f None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:40.616919 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:40.638762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 320/621] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:28:40.678752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e1809-9764-4ef4-88ef-f8e12e352e1f None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:40.732675 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ac42b-240a-4b7c-a0b8-705c3d4c5e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:40.733337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ac42b-240a-4b7c-a0b8-705c3d4c5e9b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:40.733621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ac42b-240a-4b7c-a0b8-705c3d4c5e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:40.733898 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ac42b-240a-4b7c-a0b8-705c3d4c5e9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:40.746602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ac42b-240a-4b7c-a0b8-705c3d4c5e9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:40.748668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ac42b-240a-4b7c-a0b8-705c3d4c5e9b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:40.768305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-925ac42b-240a-4b7c-a0b8-705c3d4c5e9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:40.768305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:40.768305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:40.768305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:40.768305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:40.768305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:40.768305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:40.768305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:40.768305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:40.768305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:40.768305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:40.768305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:40.768305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:40.768305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:40.768305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:40.769348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:40.771111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:40.771111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:40.771111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:40.771111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:40.771111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:40.771111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:40.771111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:40.771111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:40.771111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 302/622] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:40.775676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dd1ab-2296-46e7-a338-49de4fe827ba None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:40.793161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ac42b-240a-4b7c-a0b8-705c3d4c5e9b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:40.860036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:40.921078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dd1ab-2296-46e7-a338-49de4fe827ba None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:40.972452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ac42b-240a-4b7c-a0b8-705c3d4c5e9b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:41.009616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:41.014380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc22c7-ca10-4891-ac46-4c58758c6682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:41.015035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc22c7-ca10-4891-ac46-4c58758c6682 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:41.015320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc22c7-ca10-4891-ac46-4c58758c6682 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:41.015595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc22c7-ca10-4891-ac46-4c58758c6682 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:41.023177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f498716-5e28-48d5-9413-ea90b53e642d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:41.023832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f498716-5e28-48d5-9413-ea90b53e642d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:41.024117 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f498716-5e28-48d5-9413-ea90b53e642d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:41.024419 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f498716-5e28-48d5-9413-ea90b53e642d None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:41.025146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f498716-5e28-48d5-9413-ea90b53e642d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:41.025455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f498716-5e28-48d5-9413-ea90b53e642d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:41.026973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f498716-5e28-48d5-9413-ea90b53e642d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:41.029616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc22c7-ca10-4891-ac46-4c58758c6682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-a9dc22c7-ca10-4891-ac46-4c58758c6682 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:41.034687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:41.035699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:41.039872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:41.039872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:41.039872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:41.039872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:41.039872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:41.039872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:41.039872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:41.039872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:41.039872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:41.039872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:41.039872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:41.039872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:41.039872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 303/623] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:41.042189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc22c7-ca10-4891-ac46-4c58758c6682 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:41.057167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 321/624] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:41.060646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f498716-5e28-48d5-9413-ea90b53e642d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:41.177649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc22c7-ca10-4891-ac46-4c58758c6682 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:41.219848 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3fae0-dabb-4d29-bd3b-a014cda9c381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:41.220892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3fae0-dabb-4d29-bd3b-a014cda9c381 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:41.221091 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3fae0-dabb-4d29-bd3b-a014cda9c381 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:41.221610 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3fae0-dabb-4d29-bd3b-a014cda9c381 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:41.222553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3fae0-dabb-4d29-bd3b-a014cda9c381 None admin] RBAC: Adding query filter params (name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:41.222960 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3fae0-dabb-4d29-bd3b-a014cda9c381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:41.224501 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3fae0-dabb-4d29-bd3b-a014cda9c381 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:41.234855 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 304/625] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:41.235867 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3fae0-dabb-4d29-bd3b-a014cda9c381 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:41.236247 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f498716-5e28-48d5-9413-ea90b53e642d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:41.260817 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:41.277182 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af72c8a-a4d8-409b-bf24-8b49d7b6b538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:41.278281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af72c8a-a4d8-409b-bf24-8b49d7b6b538 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:41.283892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af72c8a-a4d8-409b-bf24-8b49d7b6b538 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:41.283892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af72c8a-a4d8-409b-bf24-8b49d7b6b538 None admin] PATH_INFO: `/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/b8ec0b1e5f4d434bb725297f3f867ba7/roles/bb34a44869834493b10a99cc751ab6c2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:41.297654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af72c8a-a4d8-409b-bf24-8b49d7b6b538 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc920dce30734ac7a69dc039d8c74ce1, user_id=b8ec0b1e5f4d434bb725297f3f867ba7, role_id=bb34a44869834493b10a99cc751ab6c2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:41.299444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af72c8a-a4d8-409b-bf24-8b49d7b6b538 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:41.300905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:41.372299 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3fae0-dabb-4d29-bd3b-a014cda9c381 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:41.373983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 322/626] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:28:41 2022] PUT /identity/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/b8ec0b1e5f4d434bb725297f3f867ba7/roles/bb34a44869834493b10a99cc751ab6c2 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:28:41.385601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af72c8a-a4d8-409b-bf24-8b49d7b6b538 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:41.453076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101fc68-7a4c-47df-93c1-cfb6055d242a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:41.454270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101fc68-7a4c-47df-93c1-cfb6055d242a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:41.454833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101fc68-7a4c-47df-93c1-cfb6055d242a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:41.455739 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101fc68-7a4c-47df-93c1-cfb6055d242a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:41.479288 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101fc68-7a4c-47df-93c1-cfb6055d242a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-4101fc68-7a4c-47df-93c1-cfb6055d242a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:41.491826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:41.492897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:41.493748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:41.493748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:41.493748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:41.493748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:41.493748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:41.493748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:41.493748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:41.493748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:41.493748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:41.493748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:41.493748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:41.493748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:41.508436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 305/627] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:41.508436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129fcf5-7af5-4f69-8e8a-aa9d1b513894 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:41.508436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129fcf5-7af5-4f69-8e8a-aa9d1b513894 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:41.508436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129fcf5-7af5-4f69-8e8a-aa9d1b513894 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:41.508812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 306/628] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:41.510355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129fcf5-7af5-4f69-8e8a-aa9d1b513894 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:41.544835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af72c8a-a4d8-409b-bf24-8b49d7b6b538 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:41.592804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6083d31-e3cf-4907-b6a0-c83f19409d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:41.593833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6083d31-e3cf-4907-b6a0-c83f19409d76 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:41.594129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6083d31-e3cf-4907-b6a0-c83f19409d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:41.594423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6083d31-e3cf-4907-b6a0-c83f19409d76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:41.623109 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6083d31-e3cf-4907-b6a0-c83f19409d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:41.624631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6083d31-e3cf-4907-b6a0-c83f19409d76 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-d6083d31-e3cf-4907-b6a0-c83f19409d76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:41.672591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:41.674487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:41.676332 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:41.676332 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:41.676332 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:41.679805 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 323/629] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:41.681856 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc2a84-496c-49f1-8af6-adcf0cbec14a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:41.682305 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc2a84-496c-49f1-8af6-adcf0cbec14a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:41.682825 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc2a84-496c-49f1-8af6-adcf0cbec14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:41.693898 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129fcf5-7af5-4f69-8e8a-aa9d1b513894 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:41.712873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:41.721673 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-14fc2a84-496c-49f1-8af6-adcf0cbec14a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:41.724091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-14fc2a84-496c-49f1-8af6-adcf0cbec14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:41.743162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:41.781017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9724213-124f-4d97-bc0d-d1a82df1987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:41.783428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9724213-124f-4d97-bc0d-d1a82df1987e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:41.784435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9724213-124f-4d97-bc0d-d1a82df1987e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:41.785317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9724213-124f-4d97-bc0d-d1a82df1987e None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:41.787178 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9724213-124f-4d97-bc0d-d1a82df1987e None admin] RBAC: Adding query filter params (name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:41.787992 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9724213-124f-4d97-bc0d-d1a82df1987e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:41.793213 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9724213-124f-4d97-bc0d-d1a82df1987e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:41.811812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 307/630] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:41.813079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9724213-124f-4d97-bc0d-d1a82df1987e None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:41.839722 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 324/631] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:41.843672 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc2a84-496c-49f1-8af6-adcf0cbec14a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:42.059338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9724213-124f-4d97-bc0d-d1a82df1987e None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:42.066473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc2a84-496c-49f1-8af6-adcf0cbec14a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:42.132178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4bee3-36a6-4be2-a6c7-3e0936b62e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:42.132392 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cb826-2e73-473c-bac3-ac7e2d3655fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:42.132890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4bee3-36a6-4be2-a6c7-3e0936b62e23 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:42.133169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4bee3-36a6-4be2-a6c7-3e0936b62e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:42.133268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cb826-2e73-473c-bac3-ac7e2d3655fb None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:42.133438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4bee3-36a6-4be2-a6c7-3e0936b62e23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:42.133731 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cb826-2e73-473c-bac3-ac7e2d3655fb None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:42.134209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cb826-2e73-473c-bac3-ac7e2d3655fb None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:42.135553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cb826-2e73-473c-bac3-ac7e2d3655fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:42.136153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cb826-2e73-473c-bac3-ac7e2d3655fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:42.139580 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cb826-2e73-473c-bac3-ac7e2d3655fb None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:42.146290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e4bee3-36a6-4be2-a6c7-3e0936b62e23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:42.147790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e4bee3-36a6-4be2-a6c7-3e0936b62e23 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:42.150129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:42.162935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 308/632] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:42.163974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cb826-2e73-473c-bac3-ac7e2d3655fb None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-c7e4bee3-36a6-4be2-a6c7-3e0936b62e23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:42.171694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:42.177356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:42.178257 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:42.178257 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:42.178257 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:42.178257 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 325/633] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:42.178257 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4bee3-36a6-4be2-a6c7-3e0936b62e23 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:42.221706 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:42.311713 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cb826-2e73-473c-bac3-ac7e2d3655fb None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:42.347674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4bee3-36a6-4be2-a6c7-3e0936b62e23 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:42.351290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0392e6-5865-4aab-8686-2b470b01f994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:42.352054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0392e6-5865-4aab-8686-2b470b01f994 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:42.352342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0392e6-5865-4aab-8686-2b470b01f994 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:42.352650 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0392e6-5865-4aab-8686-2b470b01f994 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:42.367412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0392e6-5865-4aab-8686-2b470b01f994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-7f0392e6-5865-4aab-8686-2b470b01f994 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:42.373190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:42.375373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:42.377455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:42.377455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:42.377455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:42.377455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:42.377455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:42.377455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:42.377455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:42.377455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:42.377455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:42.377455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:42.377455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:42.377455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:42.383397 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 309/634] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:42.383397 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0392e6-5865-4aab-8686-2b470b01f994 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:42.394290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513b329-bebc-4ec8-89c0-ebeac3faddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:42.395062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513b329-bebc-4ec8-89c0-ebeac3faddfe None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:42.395356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513b329-bebc-4ec8-89c0-ebeac3faddfe None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:42.395635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513b329-bebc-4ec8-89c0-ebeac3faddfe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:42.405131 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513b329-bebc-4ec8-89c0-ebeac3faddfe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:42.406679 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513b329-bebc-4ec8-89c0-ebeac3faddfe None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-a513b329-bebc-4ec8-89c0-ebeac3faddfe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:42.419754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:42.423283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:42.425584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:42.425584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:42.425584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:42.425584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:42.425584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:42.425584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:42.425584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:42.425584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:42.425584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:28:42.425584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:42.426921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 326/635] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:42.426921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513b329-bebc-4ec8-89c0-ebeac3faddfe None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:42.489526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:42.492653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:42.500964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0392e6-5865-4aab-8686-2b470b01f994 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:42.542199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbb6ea-d9ed-4a83-aa97-fe7fcf4b9a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:42.542360 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbb6ea-d9ed-4a83-aa97-fe7fcf4b9a9d None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:42.542587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbb6ea-d9ed-4a83-aa97-fe7fcf4b9a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:42.542838 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbb6ea-d9ed-4a83-aa97-fe7fcf4b9a9d None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:42.544374 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbb6ea-d9ed-4a83-aa97-fe7fcf4b9a9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:42.544374 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbb6ea-d9ed-4a83-aa97-fe7fcf4b9a9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:42.545755 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbb6ea-d9ed-4a83-aa97-fe7fcf4b9a9d None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:42.547292 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513b329-bebc-4ec8-89c0-ebeac3faddfe None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:42.566342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 310/636] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:42.566342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91fd7e-156b-4824-b6d5-923772bba41d None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:42.566342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91fd7e-156b-4824-b6d5-923772bba41d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:42.566342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91fd7e-156b-4824-b6d5-923772bba41d None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:42.566342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 311/637] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:42.567205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91fd7e-156b-4824-b6d5-923772bba41d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:42.596146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b0244-0c73-4f9f-9307-46c58dbbb686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:42.596827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b0244-0c73-4f9f-9307-46c58dbbb686 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:42.597113 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b0244-0c73-4f9f-9307-46c58dbbb686 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:42.597386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b0244-0c73-4f9f-9307-46c58dbbb686 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:42.598093 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b0244-0c73-4f9f-9307-46c58dbbb686 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:42.598444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b0244-0c73-4f9f-9307-46c58dbbb686 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:42.599854 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b0244-0c73-4f9f-9307-46c58dbbb686 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:42.609522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 327/638] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:42.610548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b0244-0c73-4f9f-9307-46c58dbbb686 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:42.705721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91fd7e-156b-4824-b6d5-923772bba41d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:42.735692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b0244-0c73-4f9f-9307-46c58dbbb686 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:42.745200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322602b-03b9-4263-8496-9eeca9c3f0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:42.745828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322602b-03b9-4263-8496-9eeca9c3f0fc None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:42.746184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322602b-03b9-4263-8496-9eeca9c3f0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:42.746462 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322602b-03b9-4263-8496-9eeca9c3f0fc None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:42.747792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322602b-03b9-4263-8496-9eeca9c3f0fc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:42.748095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322602b-03b9-4263-8496-9eeca9c3f0fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:42.749783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322602b-03b9-4263-8496-9eeca9c3f0fc None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:42.757468 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:42.775330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 312/639] 10.222.0.70 () {56 vars in 1137 bytes} [Wed Aug 24 16:28:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:42.776808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64b503-bc5b-4589-9cb3-7c433ec0c224 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:42.777097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64b503-bc5b-4589-9cb3-7c433ec0c224 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:42.777404 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64b503-bc5b-4589-9cb3-7c433ec0c224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:42.782821 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7853bb-ffd0-466d-b1cd-eb8092e1eb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:42.783889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7853bb-ffd0-466d-b1cd-eb8092e1eb4b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:42.784196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7853bb-ffd0-466d-b1cd-eb8092e1eb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:42.784509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7853bb-ffd0-466d-b1cd-eb8092e1eb4b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:42.795475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7853bb-ffd0-466d-b1cd-eb8092e1eb4b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:42.796987 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7853bb-ffd0-466d-b1cd-eb8092e1eb4b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:42.797169 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-0b64b503-bc5b-4589-9cb3-7c433ec0c224 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:42.798244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-0b64b503-bc5b-4589-9cb3-7c433ec0c224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:42.800521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-8f7853bb-ffd0-466d-b1cd-eb8092e1eb4b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:42.811112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:42.813220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:42.816938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:42.816938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:42.816938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:42.816938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:42.816938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:42.816938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:42.816938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:42.816938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:42.816938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:28:42.816938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:42.816938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 328/640] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:42.816938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7853bb-ffd0-466d-b1cd-eb8092e1eb4b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:42.914020 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 313/641] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:42.915633 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64b503-bc5b-4589-9cb3-7c433ec0c224 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:43.018395 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7853bb-ffd0-466d-b1cd-eb8092e1eb4b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:43.097866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3556f1-1155-4d08-9699-34b3687967b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:43.099348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3556f1-1155-4d08-9699-34b3687967b6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:43.100053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3556f1-1155-4d08-9699-34b3687967b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:43.100243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64b503-bc5b-4589-9cb3-7c433ec0c224 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:43.101127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3556f1-1155-4d08-9699-34b3687967b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:43.126042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3556f1-1155-4d08-9699-34b3687967b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:43.129608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3556f1-1155-4d08-9699-34b3687967b6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:43.141471 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058224e7-d065-49f0-9b0e-b8a08e88a958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:43.142104 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058224e7-d065-49f0-9b0e-b8a08e88a958 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:43.142388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058224e7-d065-49f0-9b0e-b8a08e88a958 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:43.142741 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058224e7-d065-49f0-9b0e-b8a08e88a958 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:43.152371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-ed3556f1-1155-4d08-9699-34b3687967b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:43.164937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:43.167130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:43.185461 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:43.185461 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:43.185461 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:43.185461 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 329/642] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:43.185461 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3556f1-1155-4d08-9699-34b3687967b6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:43.219721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:43.223253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058224e7-d065-49f0-9b0e-b8a08e88a958 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:43.225048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058224e7-d065-49f0-9b0e-b8a08e88a958 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:43.321800 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3556f1-1155-4d08-9699-34b3687967b6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:43.347384 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 314/643] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:43.348101 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058224e7-d065-49f0-9b0e-b8a08e88a958 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:43.374875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f04d5-41eb-49a5-917d-a5b81cf70f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:43.376083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f04d5-41eb-49a5-917d-a5b81cf70f79 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:43.376083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f04d5-41eb-49a5-917d-a5b81cf70f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:43.376383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f04d5-41eb-49a5-917d-a5b81cf70f79 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:43.376989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f04d5-41eb-49a5-917d-a5b81cf70f79 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:43.377416 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f04d5-41eb-49a5-917d-a5b81cf70f79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:43.378762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f04d5-41eb-49a5-917d-a5b81cf70f79 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:43.399516 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:43.401951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 330/644] 10.222.0.70 () {56 vars in 1137 bytes} [Wed Aug 24 16:28:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:43.402818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f04d5-41eb-49a5-917d-a5b81cf70f79 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:43.490522 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058224e7-d065-49f0-9b0e-b8a08e88a958 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:43.525500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:43.545388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd69285-0654-4ae6-9022-9a5f36757f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:43.546881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd69285-0654-4ae6-9022-9a5f36757f83 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:43.547637 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd69285-0654-4ae6-9022-9a5f36757f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:43.548519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd69285-0654-4ae6-9022-9a5f36757f83 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:43.550031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd69285-0654-4ae6-9022-9a5f36757f83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:43.550870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd69285-0654-4ae6-9022-9a5f36757f83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:43.553005 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd69285-0654-4ae6-9022-9a5f36757f83 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:43.558903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f04d5-41eb-49a5-917d-a5b81cf70f79 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:43.581830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 315/645] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:43.583925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd69285-0654-4ae6-9022-9a5f36757f83 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:43.628180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988a5a6-0a20-4fc2-8893-fc9eac9e1ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:43.628868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988a5a6-0a20-4fc2-8893-fc9eac9e1ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:43.629345 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988a5a6-0a20-4fc2-8893-fc9eac9e1ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:43.629505 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988a5a6-0a20-4fc2-8893-fc9eac9e1ce7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:43.643063 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988a5a6-0a20-4fc2-8893-fc9eac9e1ce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-1988a5a6-0a20-4fc2-8893-fc9eac9e1ce7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:43.648078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:43.653168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:43.654220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:43.654220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:43.654220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:43.654220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:43.654220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:43.654220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:43.654220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:43.654220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:43.654220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:43.654220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:43.654220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:43.654220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:43.654220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 331/646] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:43.654220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988a5a6-0a20-4fc2-8893-fc9eac9e1ce7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:43.726762 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:43.730695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd69285-0654-4ae6-9022-9a5f36757f83 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:43.773014 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988a5a6-0a20-4fc2-8893-fc9eac9e1ce7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:43.780797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bce0a8-3809-4223-aedf-9b06fc9aa384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:43.781796 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bce0a8-3809-4223-aedf-9b06fc9aa384 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:43.782246 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bce0a8-3809-4223-aedf-9b06fc9aa384 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:43.782701 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bce0a8-3809-4223-aedf-9b06fc9aa384 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:43.800882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bce0a8-3809-4223-aedf-9b06fc9aa384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:43.803166 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bce0a8-3809-4223-aedf-9b06fc9aa384 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-a7bce0a8-3809-4223-aedf-9b06fc9aa384 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:43.829170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:43.831074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:43.833181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:43.833181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:43.833181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:43.833520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b104566-6841-4177-acac-f1a6751376e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:43.833520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b104566-6841-4177-acac-f1a6751376e8 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:43.833520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b104566-6841-4177-acac-f1a6751376e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:43.833520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b104566-6841-4177-acac-f1a6751376e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:43.836585 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 316/647] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:43.839222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bce0a8-3809-4223-aedf-9b06fc9aa384 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:43.852906 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:43.921497 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b104566-6841-4177-acac-f1a6751376e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:43.922889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b104566-6841-4177-acac-f1a6751376e8 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:43.975936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bce0a8-3809-4223-aedf-9b06fc9aa384 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:44.018501 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543e81d-1581-44ca-a8f2-0e81ed9e1a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:44.019144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543e81d-1581-44ca-a8f2-0e81ed9e1a82 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:44.019421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543e81d-1581-44ca-a8f2-0e81ed9e1a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:44.019697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543e81d-1581-44ca-a8f2-0e81ed9e1a82 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:44.027710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543e81d-1581-44ca-a8f2-0e81ed9e1a82 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:44.029255 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543e81d-1581-44ca-a8f2-0e81ed9e1a82 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-f543e81d-1581-44ca-a8f2-0e81ed9e1a82 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:44.044707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:44.045746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:44.046715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:44.046715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:44.046715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:44.046715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:44.046715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:44.046715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:44.046715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:44.046715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:44.046715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:28:44.046715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:44.046715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 317/648] 10.222.0.70 () {56 vars in 1093 bytes} [Wed Aug 24 16:28:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:44.047311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543e81d-1581-44ca-a8f2-0e81ed9e1a82 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:44.063091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 332/649] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:44.063091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128ff4f-accd-4741-9089-20c5382ced0c None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:44.063091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128ff4f-accd-4741-9089-20c5382ced0c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:44.063091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128ff4f-accd-4741-9089-20c5382ced0c None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:44.063929 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 333/650] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:44.065398 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128ff4f-accd-4741-9089-20c5382ced0c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:44.067677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:44.132367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:44.192998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128ff4f-accd-4741-9089-20c5382ced0c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:44.193905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543e81d-1581-44ca-a8f2-0e81ed9e1a82 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:44.230444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a98aa-5319-49ad-a060-24f3ff844532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:44.231592 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a98aa-5319-49ad-a060-24f3ff844532 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:44.231869 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a98aa-5319-49ad-a060-24f3ff844532 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:44.232126 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a98aa-5319-49ad-a060-24f3ff844532 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:44.232817 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a98aa-5319-49ad-a060-24f3ff844532 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:44.233123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a98aa-5319-49ad-a060-24f3ff844532 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:44.234499 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a98aa-5319-49ad-a060-24f3ff844532 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:44.235462 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f987cd-92d4-4687-b28c-ff8b6cbf6134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:44.236383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f987cd-92d4-4687-b28c-ff8b6cbf6134 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:44.236743 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f987cd-92d4-4687-b28c-ff8b6cbf6134 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:44.237632 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f987cd-92d4-4687-b28c-ff8b6cbf6134 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:44.238487 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f987cd-92d4-4687-b28c-ff8b6cbf6134 None admin] RBAC: Adding query filter params (name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:44.239040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f987cd-92d4-4687-b28c-ff8b6cbf6134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:44.242407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f987cd-92d4-4687-b28c-ff8b6cbf6134 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:44.247804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 334/651] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:44.248691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a98aa-5319-49ad-a060-24f3ff844532 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:44.263001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 318/652] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:44.263001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f987cd-92d4-4687-b28c-ff8b6cbf6134 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:44.339601 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:44.372793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f987cd-92d4-4687-b28c-ff8b6cbf6134 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:44.384170 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a98aa-5319-49ad-a060-24f3ff844532 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:44.397835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:44.418135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc4f76-07f2-4562-83b3-e9b2587ff6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:44.419649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc4f76-07f2-4562-83b3-e9b2587ff6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:44.420002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc4f76-07f2-4562-83b3-e9b2587ff6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:44.420483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc4f76-07f2-4562-83b3-e9b2587ff6d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:44.437281 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc4f76-07f2-4562-83b3-e9b2587ff6d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:44.438982 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc4f76-07f2-4562-83b3-e9b2587ff6d3 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:44.439543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751aeb3-188d-49d1-aa06-9b696af5112c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:44.440156 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751aeb3-188d-49d1-aa06-9b696af5112c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:44.440474 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751aeb3-188d-49d1-aa06-9b696af5112c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:44.440748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751aeb3-188d-49d1-aa06-9b696af5112c None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:44.441500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751aeb3-188d-49d1-aa06-9b696af5112c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:44.441795 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751aeb3-188d-49d1-aa06-9b696af5112c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:44.443282 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751aeb3-188d-49d1-aa06-9b696af5112c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-9bbc4f76-07f2-4562-83b3-e9b2587ff6d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:44.456266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:44.457316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:44.458189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:44.458189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:44.458189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:44.458189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 319/653] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:44.458689 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd883ce1-a5d7-473b-ba10-443aaba7a0ce None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:44.458886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd883ce1-a5d7-473b-ba10-443aaba7a0ce None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:44.459137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd883ce1-a5d7-473b-ba10-443aaba7a0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:44.467980 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 335/654] 10.222.0.70 () {56 vars in 1133 bytes} [Wed Aug 24 16:28:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:44.469074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751aeb3-188d-49d1-aa06-9b696af5112c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:44.478209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-cd883ce1-a5d7-473b-ba10-443aaba7a0ce None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:44.479396 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-cd883ce1-a5d7-473b-ba10-443aaba7a0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:44.562812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 320/655] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:44.563798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd883ce1-a5d7-473b-ba10-443aaba7a0ce None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:44.607868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751aeb3-188d-49d1-aa06-9b696af5112c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:44.654343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:44.659025 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d0482-4d8c-48b8-ab9b-04e714214f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:44.660137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d0482-4d8c-48b8-ab9b-04e714214f3a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:44.660678 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d0482-4d8c-48b8-ab9b-04e714214f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:44.665755 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d0482-4d8c-48b8-ab9b-04e714214f3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:44.670782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d0482-4d8c-48b8-ab9b-04e714214f3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:44.671803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d0482-4d8c-48b8-ab9b-04e714214f3a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:44.690253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-ea5d0482-4d8c-48b8-ab9b-04e714214f3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:44.690253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:44.690253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:44.690253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:44.690253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:44.690253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:44.690253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:44.690253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:44.690253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:44.690253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:44.690253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:44.690253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:44.690253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:44.690253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:44.690253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:44.691489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:44.692827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:44.692827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:44.692827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:44.692827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:44.692827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:44.692827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:44.692827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:44.692827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:44.692827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 336/656] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:44.692827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d0482-4d8c-48b8-ab9b-04e714214f3a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:44.722371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd883ce1-a5d7-473b-ba10-443aaba7a0ce None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:44.734058 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:44.784528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f5b06-8c29-4525-bf9e-1a5bdf7afa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:44.785630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f5b06-8c29-4525-bf9e-1a5bdf7afa2e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:44.786127 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f5b06-8c29-4525-bf9e-1a5bdf7afa2e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:44.786621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f5b06-8c29-4525-bf9e-1a5bdf7afa2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:44.800915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f5b06-8c29-4525-bf9e-1a5bdf7afa2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:44.802842 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f5b06-8c29-4525-bf9e-1a5bdf7afa2e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-6a5f5b06-8c29-4525-bf9e-1a5bdf7afa2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:44.820840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:44.821881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:44.822756 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:44.822756 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:44.822756 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:44.822756 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 321/657] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:44.822987 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f5b06-8c29-4525-bf9e-1a5bdf7afa2e None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:44.833320 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d0482-4d8c-48b8-ab9b-04e714214f3a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:44.872370 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1e3c6-497e-4b46-af2e-27a2d82e4544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:44.873053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1e3c6-497e-4b46-af2e-27a2d82e4544 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:44.873330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1e3c6-497e-4b46-af2e-27a2d82e4544 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:44.873606 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1e3c6-497e-4b46-af2e-27a2d82e4544 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:44.874474 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1e3c6-497e-4b46-af2e-27a2d82e4544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:44.874778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1e3c6-497e-4b46-af2e-27a2d82e4544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:44.876188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1e3c6-497e-4b46-af2e-27a2d82e4544 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:44.888797 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 337/658] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:44.889662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1e3c6-497e-4b46-af2e-27a2d82e4544 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:44.950637 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f5b06-8c29-4525-bf9e-1a5bdf7afa2e None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:44.957875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:44.982247 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:44.986648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbfc95c-82b8-4b3d-8268-6f2ff8f90595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:44.987283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfc95c-82b8-4b3d-8268-6f2ff8f90595 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:44.987558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfc95c-82b8-4b3d-8268-6f2ff8f90595 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:44.987837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfc95c-82b8-4b3d-8268-6f2ff8f90595 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:44.998558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1e3c6-497e-4b46-af2e-27a2d82e4544 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:45.042916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8659c20-678e-4350-86c8-9abd57ff14ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:45.043892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8659c20-678e-4350-86c8-9abd57ff14ec None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:45.044356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8659c20-678e-4350-86c8-9abd57ff14ec None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:45.045057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8659c20-678e-4350-86c8-9abd57ff14ec None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:45.046220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8659c20-678e-4350-86c8-9abd57ff14ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:45.046667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8659c20-678e-4350-86c8-9abd57ff14ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:45.049221 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8659c20-678e-4350-86c8-9abd57ff14ec None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:45.067283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 338/659] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:45.069342 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8659c20-678e-4350-86c8-9abd57ff14ec None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:45.080275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfc95c-82b8-4b3d-8268-6f2ff8f90595 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:45.081784 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfc95c-82b8-4b3d-8268-6f2ff8f90595 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:45.187959 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8659c20-678e-4350-86c8-9abd57ff14ec None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:45.202280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 322/660] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:45.203170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbfc95c-82b8-4b3d-8268-6f2ff8f90595 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:45.213722 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:45.237295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299359aa-5f93-4d51-b6ec-9457c75ec276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:45.237836 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299359aa-5f93-4d51-b6ec-9457c75ec276 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:45.238160 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299359aa-5f93-4d51-b6ec-9457c75ec276 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:45.238432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299359aa-5f93-4d51-b6ec-9457c75ec276 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:45.239097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299359aa-5f93-4d51-b6ec-9457c75ec276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:45.239402 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299359aa-5f93-4d51-b6ec-9457c75ec276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:45.240842 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299359aa-5f93-4d51-b6ec-9457c75ec276 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:45.256145 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 339/661] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:45.256145 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299359aa-5f93-4d51-b6ec-9457c75ec276 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:45.259520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:45.339040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbfc95c-82b8-4b3d-8268-6f2ff8f90595 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:45.362431 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299359aa-5f93-4d51-b6ec-9457c75ec276 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:45.375611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbafc1-206a-4217-9cf9-3f49283915c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:45.376255 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbafc1-206a-4217-9cf9-3f49283915c4 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:45.376569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbafc1-206a-4217-9cf9-3f49283915c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:45.376844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbafc1-206a-4217-9cf9-3f49283915c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:45.389060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbafc1-206a-4217-9cf9-3f49283915c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-83bbafc1-206a-4217-9cf9-3f49283915c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:45.394200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:45.395280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:45.396857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:45.396857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:45.396857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:45.396857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:45.396857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:45.396857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:45.396857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:45.396857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:45.396857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:45.396857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:45.396857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:45.396857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:45.396857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 323/662] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:45.396857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbafc1-206a-4217-9cf9-3f49283915c4 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:45.401700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2f57a-c3ea-404b-978e-d9e565e35dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:45.403667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2f57a-c3ea-404b-978e-d9e565e35dfa None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:45.404021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2f57a-c3ea-404b-978e-d9e565e35dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:45.404307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2f57a-c3ea-404b-978e-d9e565e35dfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:45.414011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2f57a-c3ea-404b-978e-d9e565e35dfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:45.415584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2f57a-c3ea-404b-978e-d9e565e35dfa None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:45.427801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-23f2f57a-c3ea-404b-978e-d9e565e35dfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:45.427801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:45.427801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:45.427801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:45.427801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:45.427801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:45.427801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:45.427801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:45.427801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:45.427801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:45.427801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:45.427801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:45.427801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:45.427801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:45.427801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:45.428833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:45.430463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:45.430463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:45.430463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:45.430463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:45.430463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:45.430463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:45.430463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:45.430463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:45.430463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 340/663] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:45.430463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2f57a-c3ea-404b-978e-d9e565e35dfa None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:45.482262 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:45.506617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:45.532886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbafc1-206a-4217-9cf9-3f49283915c4 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:45.554223 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2f57a-c3ea-404b-978e-d9e565e35dfa None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:45.575225 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023242b-167b-4cb7-959c-aa01c2fa02c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:45.575850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023242b-167b-4cb7-959c-aa01c2fa02c0 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:45.576146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023242b-167b-4cb7-959c-aa01c2fa02c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:45.576429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023242b-167b-4cb7-959c-aa01c2fa02c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:45.577175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023242b-167b-4cb7-959c-aa01c2fa02c0 None admin] RBAC: Adding query filter params (role.id=bb34a44869834493b10a99cc751ab6c2, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=f08140bcbd1e43dda77d4387666f9085) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:45.577485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023242b-167b-4cb7-959c-aa01c2fa02c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:45.578965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023242b-167b-4cb7-959c-aa01c2fa02c0 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:45.585713 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 324/664] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:45 2022] GET /identity/v3/role_assignments?user.id=f08140bcbd1e43dda77d4387666f9085&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=bb34a44869834493b10a99cc751ab6c2 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:45.586598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023242b-167b-4cb7-959c-aa01c2fa02c0 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:45.600197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea494c-8ab5-4c20-b327-b9859877edd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:45.601538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea494c-8ab5-4c20-b327-b9859877edd2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:45.601966 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea494c-8ab5-4c20-b327-b9859877edd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:45.602608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea494c-8ab5-4c20-b327-b9859877edd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:45.624562 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea494c-8ab5-4c20-b327-b9859877edd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:45.626310 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea494c-8ab5-4c20-b327-b9859877edd2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-d8ea494c-8ab5-4c20-b327-b9859877edd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:45.644077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:45.645306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:45.646374 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:45.646374 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:45.646374 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:45.652866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 341/665] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:45.654229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea494c-8ab5-4c20-b327-b9859877edd2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:45.753393 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023242b-167b-4cb7-959c-aa01c2fa02c0 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:45.794923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea494c-8ab5-4c20-b327-b9859877edd2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:45.797116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:45.804193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f4aab-a11e-4716-a64e-6967a6ecd3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:45.804899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f4aab-a11e-4716-a64e-6967a6ecd3cc None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:45.805183 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f4aab-a11e-4716-a64e-6967a6ecd3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:45.805458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f4aab-a11e-4716-a64e-6967a6ecd3cc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:45.814878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f4aab-a11e-4716-a64e-6967a6ecd3cc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:45.816339 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f4aab-a11e-4716-a64e-6967a6ecd3cc None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:45.821438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-7b9f4aab-a11e-4716-a64e-6967a6ecd3cc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:45.831608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:45.834201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:45.836303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:45.836303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:45.836303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:45.836303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:45.836303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:45.836303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:45.836303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:45.836303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:45.836303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:28:45.836303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:45.836303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 325/666] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:45.836303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f4aab-a11e-4716-a64e-6967a6ecd3cc None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:45.854264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e7828-1d47-408a-a93d-45b1cd6122be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:45.854891 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e7828-1d47-408a-a93d-45b1cd6122be None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:45.855210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e7828-1d47-408a-a93d-45b1cd6122be None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:45.855497 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e7828-1d47-408a-a93d-45b1cd6122be None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:45.856827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e7828-1d47-408a-a93d-45b1cd6122be None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:45.857135 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e7828-1d47-408a-a93d-45b1cd6122be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:45.858557 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e7828-1d47-408a-a93d-45b1cd6122be None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:45.871412 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 342/667] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:45.876721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e7828-1d47-408a-a93d-45b1cd6122be None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:45.985852 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f4aab-a11e-4716-a64e-6967a6ecd3cc None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:46.012319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e7828-1d47-408a-a93d-45b1cd6122be None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:46.043297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0cd97-2145-48b0-9e59-651abf31e75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:46.044635 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0cd97-2145-48b0-9e59-651abf31e75b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:46.045047 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0cd97-2145-48b0-9e59-651abf31e75b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:46.046887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0cd97-2145-48b0-9e59-651abf31e75b None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:46.046887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0cd97-2145-48b0-9e59-651abf31e75b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:46.046887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0cd97-2145-48b0-9e59-651abf31e75b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:46.048542 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0cd97-2145-48b0-9e59-651abf31e75b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:46.063003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 326/668] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:46.063712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0cd97-2145-48b0-9e59-651abf31e75b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:46.076455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec28d9-7560-4821-b9b8-2cfab81bc0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:46.077482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec28d9-7560-4821-b9b8-2cfab81bc0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:46.077876 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec28d9-7560-4821-b9b8-2cfab81bc0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:46.078152 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec28d9-7560-4821-b9b8-2cfab81bc0c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:46.079317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec28d9-7560-4821-b9b8-2cfab81bc0c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:46.079962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec28d9-7560-4821-b9b8-2cfab81bc0c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:46.082471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec28d9-7560-4821-b9b8-2cfab81bc0c3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:46.104614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 343/669] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:46.105539 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec28d9-7560-4821-b9b8-2cfab81bc0c3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:46.153847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:46.155695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:46.226412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0cd97-2145-48b0-9e59-651abf31e75b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:46.241567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec28d9-7560-4821-b9b8-2cfab81bc0c3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:46.297651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c67d2-a6dc-4f9a-9367-8c79fe5d081c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:46.298625 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c67d2-a6dc-4f9a-9367-8c79fe5d081c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:46.299123 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c67d2-a6dc-4f9a-9367-8c79fe5d081c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:46.299621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c67d2-a6dc-4f9a-9367-8c79fe5d081c None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:46.301081 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c67d2-a6dc-4f9a-9367-8c79fe5d081c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:46.301788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c67d2-a6dc-4f9a-9367-8c79fe5d081c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:46.304494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c67d2-a6dc-4f9a-9367-8c79fe5d081c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:46.314539 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfd6d2-4993-4d47-a895-e6233c65a595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:46.315644 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfd6d2-4993-4d47-a895-e6233c65a595 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:46.316148 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfd6d2-4993-4d47-a895-e6233c65a595 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:46.316693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfd6d2-4993-4d47-a895-e6233c65a595 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:46.337679 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfd6d2-4993-4d47-a895-e6233c65a595 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:46.344225 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 327/670] 10.222.0.70 () {56 vars in 1139 bytes} [Wed Aug 24 16:28:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:46.344804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfd6d2-4993-4d47-a895-e6233c65a595 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:46.344910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c67d2-a6dc-4f9a-9367-8c79fe5d081c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-19dfd6d2-4993-4d47-a895-e6233c65a595 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:46.361372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:46.370875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:46.372281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:46.372281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:46.372281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:46.372281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 344/671] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:46.372281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfd6d2-4993-4d47-a895-e6233c65a595 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:46.499535 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:46.504191 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c67d2-a6dc-4f9a-9367-8c79fe5d081c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:46.511705 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfd6d2-4993-4d47-a895-e6233c65a595 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:46.517003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:46.549970 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a173ded-10a5-482b-ba20-cae27d37d4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:46.551210 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a173ded-10a5-482b-ba20-cae27d37d4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:46.551812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a173ded-10a5-482b-ba20-cae27d37d4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:46.551920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a173ded-10a5-482b-ba20-cae27d37d4e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:46.552798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a173ded-10a5-482b-ba20-cae27d37d4e7 None admin] RBAC: Adding query filter params (role.id=bb34a44869834493b10a99cc751ab6c2, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=56418ff737504181a59714aad9f1c375) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:46.553085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a173ded-10a5-482b-ba20-cae27d37d4e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:46.554484 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a173ded-10a5-482b-ba20-cae27d37d4e7 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:46.569010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 328/672] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:46 2022] GET /identity/v3/role_assignments?user.id=56418ff737504181a59714aad9f1c375&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=bb34a44869834493b10a99cc751ab6c2 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:46.569010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a173ded-10a5-482b-ba20-cae27d37d4e7 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:46.569410 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d83a3-ad1c-4443-8f19-2621fa5acf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:46.569410 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d83a3-ad1c-4443-8f19-2621fa5acf8d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:46.569410 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d83a3-ad1c-4443-8f19-2621fa5acf8d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:46.569410 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d83a3-ad1c-4443-8f19-2621fa5acf8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:46.583091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d83a3-ad1c-4443-8f19-2621fa5acf8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:46.584775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d83a3-ad1c-4443-8f19-2621fa5acf8d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:46.598432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-698d83a3-ad1c-4443-8f19-2621fa5acf8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:46.598432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:46.598432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:46.598432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:46.598432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:46.598432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:46.598432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:46.598432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:46.598432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:46.598432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:46.598432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:46.598432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:46.598432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:46.598432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:46.598432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:46.599399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:46.600289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:46.600289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:46.600289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:46.600289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:46.600289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:46.600289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:46.600289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:46.600289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:46.600289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 345/673] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:46.601131 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d83a3-ad1c-4443-8f19-2621fa5acf8d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:46.746243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a173ded-10a5-482b-ba20-cae27d37d4e7 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:46.801731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d83a3-ad1c-4443-8f19-2621fa5acf8d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:46.807089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715c885-d81e-4d85-87d3-ac8ade419226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:46.807089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715c885-d81e-4d85-87d3-ac8ade419226 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:46.807593 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715c885-d81e-4d85-87d3-ac8ade419226 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:46.807874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715c885-d81e-4d85-87d3-ac8ade419226 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:46.847737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd508a5e-884a-4eb8-8f2b-fa4086907c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:46.848362 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd508a5e-884a-4eb8-8f2b-fa4086907c3b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:46.848692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd508a5e-884a-4eb8-8f2b-fa4086907c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:46.848967 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd508a5e-884a-4eb8-8f2b-fa4086907c3b None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:46.849688 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd508a5e-884a-4eb8-8f2b-fa4086907c3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:46.849996 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd508a5e-884a-4eb8-8f2b-fa4086907c3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:46.851413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd508a5e-884a-4eb8-8f2b-fa4086907c3b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:46.865438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 346/674] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:46.866348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd508a5e-884a-4eb8-8f2b-fa4086907c3b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:46.885766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:46.888439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:46.955158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d715c885-d81e-4d85-87d3-ac8ade419226 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:46.958806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d715c885-d81e-4d85-87d3-ac8ade419226 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:47.020792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd508a5e-884a-4eb8-8f2b-fa4086907c3b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:47.102052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf8445-d121-483b-bdb6-3d072ae1738d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:47.103292 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf8445-d121-483b-bdb6-3d072ae1738d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:47.103570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf8445-d121-483b-bdb6-3d072ae1738d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:47.104043 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf8445-d121-483b-bdb6-3d072ae1738d None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:47.105333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf8445-d121-483b-bdb6-3d072ae1738d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:47.106537 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf8445-d121-483b-bdb6-3d072ae1738d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:47.109224 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf8445-d121-483b-bdb6-3d072ae1738d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:47.135473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 347/675] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:47.137677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf8445-d121-483b-bdb6-3d072ae1738d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:47.184681 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 329/676] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:47.186389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715c885-d81e-4d85-87d3-ac8ade419226 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:47.288396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:47.332079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:47.332455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf8445-d121-483b-bdb6-3d072ae1738d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:47.372420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c1b9c-7135-41d4-a517-a272be2d27c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:47.373505 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c1b9c-7135-41d4-a517-a272be2d27c4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:47.373990 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c1b9c-7135-41d4-a517-a272be2d27c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:47.374476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c1b9c-7135-41d4-a517-a272be2d27c4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:47.383420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c1b9c-7135-41d4-a517-a272be2d27c4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:47.385280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c1b9c-7135-41d4-a517-a272be2d27c4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:47.394297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715c885-d81e-4d85-87d3-ac8ade419226 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-ce2c1b9c-7135-41d4-a517-a272be2d27c4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:47.399726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:47.400756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:47.401700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:47.401700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:47.401700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:47.401700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:47.401700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:47.401700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:47.401700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:47.401700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:47.401700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:28:47.401700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:47.404339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 348/677] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:47.405465 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c1b9c-7135-41d4-a517-a272be2d27c4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:47.441643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd047554-6d2b-43a9-823d-37ca93cb38be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:47.442749 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd047554-6d2b-43a9-823d-37ca93cb38be None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:47.443213 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd047554-6d2b-43a9-823d-37ca93cb38be None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:47.443712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd047554-6d2b-43a9-823d-37ca93cb38be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:47.445223 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd047554-6d2b-43a9-823d-37ca93cb38be None admin] RBAC: Adding query filter params (role.id=bb34a44869834493b10a99cc751ab6c2, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=e333fca7790244f6a4ebceabb55faba4) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:47.445807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd047554-6d2b-43a9-823d-37ca93cb38be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:47.448602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd047554-6d2b-43a9-823d-37ca93cb38be None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:47.464907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 330/678] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:47 2022] GET /identity/v3/role_assignments?user.id=e333fca7790244f6a4ebceabb55faba4&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=bb34a44869834493b10a99cc751ab6c2 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:47.466485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd047554-6d2b-43a9-823d-37ca93cb38be None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:47.545837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c1b9c-7135-41d4-a517-a272be2d27c4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:47.641400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:47.648677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d25ae1-1ac8-4b7a-abc9-93a9db7faaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:47.649333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d25ae1-1ac8-4b7a-abc9-93a9db7faaa0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:47.649664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d25ae1-1ac8-4b7a-abc9-93a9db7faaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:47.649939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d25ae1-1ac8-4b7a-abc9-93a9db7faaa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:47.670023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d25ae1-1ac8-4b7a-abc9-93a9db7faaa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:47.671544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d25ae1-1ac8-4b7a-abc9-93a9db7faaa0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:47.705923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd047554-6d2b-43a9-823d-37ca93cb38be None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-40d25ae1-1ac8-4b7a-abc9-93a9db7faaa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:47.718941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:47.720567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:47.722247 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:47.722247 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:47.722247 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:47.725732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 349/679] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:47.738463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d25ae1-1ac8-4b7a-abc9-93a9db7faaa0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:47.790892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe4879-14cc-44cd-9114-971bb5bda8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:47.791565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe4879-14cc-44cd-9114-971bb5bda8da None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:47.791848 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe4879-14cc-44cd-9114-971bb5bda8da None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:47.792119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe4879-14cc-44cd-9114-971bb5bda8da None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:47.794358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe4879-14cc-44cd-9114-971bb5bda8da None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:47.794358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe4879-14cc-44cd-9114-971bb5bda8da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:47.797114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe4879-14cc-44cd-9114-971bb5bda8da None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:47.801915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:47.834125 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 331/680] 10.222.0.70 () {56 vars in 1135 bytes} [Wed Aug 24 16:28:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:47.836076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea9a76-62dc-4174-a987-ef9479a276dd None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:47.836917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea9a76-62dc-4174-a987-ef9479a276dd None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:47.837514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea9a76-62dc-4174-a987-ef9479a276dd None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:47.842512 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 332/681] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:47.843826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea9a76-62dc-4174-a987-ef9479a276dd None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:47.909874 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d25ae1-1ac8-4b7a-abc9-93a9db7faaa0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:47.958807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034d256-38b3-4d3a-aa66-80398ec8cf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:47.959850 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034d256-38b3-4d3a-aa66-80398ec8cf7f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:47.960373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034d256-38b3-4d3a-aa66-80398ec8cf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:47.960900 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034d256-38b3-4d3a-aa66-80398ec8cf7f None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:47.961813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034d256-38b3-4d3a-aa66-80398ec8cf7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:47.962340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034d256-38b3-4d3a-aa66-80398ec8cf7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:47.963963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034d256-38b3-4d3a-aa66-80398ec8cf7f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:47.978483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 350/682] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:47.980322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac137b8d-ff1f-40f7-851d-75506fb9db07 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:47.981434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac137b8d-ff1f-40f7-851d-75506fb9db07 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:47.981878 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac137b8d-ff1f-40f7-851d-75506fb9db07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:47.998470 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea9a76-62dc-4174-a987-ef9479a276dd None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:48.014388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ac137b8d-ff1f-40f7-851d-75506fb9db07 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:48.016295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ac137b8d-ff1f-40f7-851d-75506fb9db07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:48.057239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fa6df-d650-4f4c-abd4-0ab9f3e49864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:48.057971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:48.059749 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fa6df-d650-4f4c-abd4-0ab9f3e49864 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:48.060338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fa6df-d650-4f4c-abd4-0ab9f3e49864 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:48.060787 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fa6df-d650-4f4c-abd4-0ab9f3e49864 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:48.104561 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 351/683] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:48.105623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac137b8d-ff1f-40f7-851d-75506fb9db07 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:48.155054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:48.164604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fa6df-d650-4f4c-abd4-0ab9f3e49864 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:48.166355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fa6df-d650-4f4c-abd4-0ab9f3e49864 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:48.324213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac137b8d-ff1f-40f7-851d-75506fb9db07 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:48.370882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 333/684] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:48.376318 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fa6df-d650-4f4c-abd4-0ab9f3e49864 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:48.405550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81710c3e-9e78-4160-8816-90fb9ab3d223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:48.407838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81710c3e-9e78-4160-8816-90fb9ab3d223 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:48.408634 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81710c3e-9e78-4160-8816-90fb9ab3d223 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:48.409599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81710c3e-9e78-4160-8816-90fb9ab3d223 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:48.426159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81710c3e-9e78-4160-8816-90fb9ab3d223 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:48.430406 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81710c3e-9e78-4160-8816-90fb9ab3d223 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:48.456053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-81710c3e-9e78-4160-8816-90fb9ab3d223 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:48.456053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:48.456053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:48.456053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:48.456053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:48.456053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:48.456053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:48.456053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:48.456053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:48.456053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:48.456053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:48.456053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:48.456053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:48.456053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:48.456053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:48.457806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:48.460815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:48.460815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:48.460815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:48.460815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:48.460815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:48.460815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:48.460815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:48.460815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:48.465143 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 352/685] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:48.475741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81710c3e-9e78-4160-8816-90fb9ab3d223 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:48.555482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:48.593874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:48.603674 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fa6df-d650-4f4c-abd4-0ab9f3e49864 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:48.658291 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12f12b-86bc-4a92-8ccb-efeb32831550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:48.658971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12f12b-86bc-4a92-8ccb-efeb32831550 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:48.659253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12f12b-86bc-4a92-8ccb-efeb32831550 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:48.659521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12f12b-86bc-4a92-8ccb-efeb32831550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:48.674983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12f12b-86bc-4a92-8ccb-efeb32831550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:48.677018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12f12b-86bc-4a92-8ccb-efeb32831550 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-5d12f12b-86bc-4a92-8ccb-efeb32831550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:48.701097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:48.702216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:48.703586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:48.703586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:48.703586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:48.703586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 334/686] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:28:48.703586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12f12b-86bc-4a92-8ccb-efeb32831550 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:48.705608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81710c3e-9e78-4160-8816-90fb9ab3d223 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:48.750616 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd907563-11eb-473f-adda-18c6d0e30140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:48.751633 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd907563-11eb-473f-adda-18c6d0e30140 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:48.752129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd907563-11eb-473f-adda-18c6d0e30140 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:48.752710 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd907563-11eb-473f-adda-18c6d0e30140 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:48.753807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd907563-11eb-473f-adda-18c6d0e30140 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:48.754295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd907563-11eb-473f-adda-18c6d0e30140 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:48.755950 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd907563-11eb-473f-adda-18c6d0e30140 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:48.770948 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 353/687] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:28:48.772090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd907563-11eb-473f-adda-18c6d0e30140 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:48.898871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12f12b-86bc-4a92-8ccb-efeb32831550 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:48.954098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:48.967795 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd907563-11eb-473f-adda-18c6d0e30140 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:48.976591 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95201f-f6f9-4d78-a69e-971e97bd19ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:48.978156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95201f-f6f9-4d78-a69e-971e97bd19ec None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:48.978929 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95201f-f6f9-4d78-a69e-971e97bd19ec None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:48.979808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95201f-f6f9-4d78-a69e-971e97bd19ec None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:48.981437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d95201f-f6f9-4d78-a69e-971e97bd19ec None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:48.984446 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d95201f-f6f9-4d78-a69e-971e97bd19ec None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-7d95201f-f6f9-4d78-a69e-971e97bd19ec None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:49.009640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:49.011054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:49.012144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:49.012144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:49.012144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:49.012144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:49.012144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:49.012144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:49.012144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:49.012144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:49.012144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:49.012144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:49.012144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:28:49.012144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:28:49.012144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:28:49.012144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:49.013072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4a359-5989-45f9-8230-9d7c6600613a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:49.013072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4a359-5989-45f9-8230-9d7c6600613a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:49.013072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4a359-5989-45f9-8230-9d7c6600613a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:49.013072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4a359-5989-45f9-8230-9d7c6600613a None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:49.013366 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4a359-5989-45f9-8230-9d7c6600613a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:49.013366 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4a359-5989-45f9-8230-9d7c6600613a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:49.014658 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4a359-5989-45f9-8230-9d7c6600613a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:49.019175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 335/688] 10.222.0.70 () {56 vars in 1129 bytes} [Wed Aug 24 16:28:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:49.022473 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95201f-f6f9-4d78-a69e-971e97bd19ec None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:49.036793 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 354/689] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:49.038259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4a359-5989-45f9-8230-9d7c6600613a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:49.038932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:49.181422 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4a359-5989-45f9-8230-9d7c6600613a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:49.193172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95201f-f6f9-4d78-a69e-971e97bd19ec None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:49.230994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c7fa6-e960-43ba-9be9-8f01292718c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:49.231522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c7fa6-e960-43ba-9be9-8f01292718c3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:49.231814 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c7fa6-e960-43ba-9be9-8f01292718c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:49.232084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c7fa6-e960-43ba-9be9-8f01292718c3 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:49.232819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c7fa6-e960-43ba-9be9-8f01292718c3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:49.233124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c7fa6-e960-43ba-9be9-8f01292718c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:49.234504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c7fa6-e960-43ba-9be9-8f01292718c3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:49.239361 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 355/690] 10.222.0.70 () {56 vars in 1124 bytes} [Wed Aug 24 16:28:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:49.240208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c7fa6-e960-43ba-9be9-8f01292718c3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:49.253622 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46727220-d0dc-432f-86e6-32d838de1144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:49.254583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46727220-d0dc-432f-86e6-32d838de1144 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:49.254874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46727220-d0dc-432f-86e6-32d838de1144 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:49.255344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46727220-d0dc-432f-86e6-32d838de1144 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:49.256108 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46727220-d0dc-432f-86e6-32d838de1144 None admin] RBAC: Adding query filter params (role.id=bb34a44869834493b10a99cc751ab6c2, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=942b54c39ae5403c8ae3bcca1c31bf1a) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:49.256440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46727220-d0dc-432f-86e6-32d838de1144 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:49.257898 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46727220-d0dc-432f-86e6-32d838de1144 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:49.262958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:49.274335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 336/691] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:49 2022] GET /identity/v3/role_assignments?user.id=942b54c39ae5403c8ae3bcca1c31bf1a&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=bb34a44869834493b10a99cc751ab6c2 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:49.274335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea8e53-c866-4b2a-b1e2-17a03fc3b261 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:49.274335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea8e53-c866-4b2a-b1e2-17a03fc3b261 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:49.274335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea8e53-c866-4b2a-b1e2-17a03fc3b261 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:49.274335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 337/692] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:49.274335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea8e53-c866-4b2a-b1e2-17a03fc3b261 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:49.414627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c7fa6-e960-43ba-9be9-8f01292718c3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:49.442321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:49.478087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea8e53-c866-4b2a-b1e2-17a03fc3b261 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:49.489865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c539090-7afb-4980-8f96-df88f9bc6469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:49.491059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c539090-7afb-4980-8f96-df88f9bc6469 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:49.491568 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c539090-7afb-4980-8f96-df88f9bc6469 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:49.492231 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c539090-7afb-4980-8f96-df88f9bc6469 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:49.505347 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c539090-7afb-4980-8f96-df88f9bc6469 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:49.508183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c539090-7afb-4980-8f96-df88f9bc6469 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:49.523745 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a712c36-80cd-4a2b-a66d-e2d5c55edee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:49.524370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a712c36-80cd-4a2b-a66d-e2d5c55edee8 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:49.524690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a712c36-80cd-4a2b-a66d-e2d5c55edee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:49.524965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a712c36-80cd-4a2b-a66d-e2d5c55edee8 None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:49.525620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a712c36-80cd-4a2b-a66d-e2d5c55edee8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:49.525930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a712c36-80cd-4a2b-a66d-e2d5c55edee8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:49.527444 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a712c36-80cd-4a2b-a66d-e2d5c55edee8 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:49.533978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-3c539090-7afb-4980-8f96-df88f9bc6469 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:49.533978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:49.533978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:49.533978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:49.533978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:49.533978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:49.533978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:49.533978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:49.533978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:49.533978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:49.533978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:49.533978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:49.533978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:49.533978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:49.533978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:49.534960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:49.535844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:49.535844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:49.535844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:49.535844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:49.535844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:49.535844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:49.535844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:49.535844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:49.537233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 356/694] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:49.537370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 338/694] 10.222.0.70 () {56 vars in 1124 bytes} [Wed Aug 24 16:28:49 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:49.541401 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd082d-a3b0-4089-8192-b5b0f03bd91c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:49.541923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd082d-a3b0-4089-8192-b5b0f03bd91c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:49.542430 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd082d-a3b0-4089-8192-b5b0f03bd91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:49.545082 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a712c36-80cd-4a2b-a66d-e2d5c55edee8 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:49.577992 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-83cd082d-a3b0-4089-8192-b5b0f03bd91c None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:49.580185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-83cd082d-a3b0-4089-8192-b5b0f03bd91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:49.702727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 357/695] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:49.745012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a712c36-80cd-4a2b-a66d-e2d5c55edee8 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:49.790822 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0e6e7-57b0-4e2a-a20a-56a4972ece6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:49.791470 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0e6e7-57b0-4e2a-a20a-56a4972ece6a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:49.791757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0e6e7-57b0-4e2a-a20a-56a4972ece6a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:49.792034 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0e6e7-57b0-4e2a-a20a-56a4972ece6a None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:49.794303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0e6e7-57b0-4e2a-a20a-56a4972ece6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:49.794617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0e6e7-57b0-4e2a-a20a-56a4972ece6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:49.796284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0e6e7-57b0-4e2a-a20a-56a4972ece6a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:49.805879 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:49.815435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 339/696] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:28:49.822098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd082d-a3b0-4089-8192-b5b0f03bd91c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:49.848912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:49.975045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd082d-a3b0-4089-8192-b5b0f03bd91c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:50.024506 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594461e1-de99-4874-baba-224a76231a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:50.025091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594461e1-de99-4874-baba-224a76231a0b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:50.025372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594461e1-de99-4874-baba-224a76231a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:50.025652 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594461e1-de99-4874-baba-224a76231a0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:50.026410 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594461e1-de99-4874-baba-224a76231a0b None admin] RBAC: Adding query filter params (role.id=bb34a44869834493b10a99cc751ab6c2, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=b8ec0b1e5f4d434bb725297f3f867ba7) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:50.026718 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594461e1-de99-4874-baba-224a76231a0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:50.028363 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594461e1-de99-4874-baba-224a76231a0b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:50.035663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 358/697] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:49 2022] GET /identity/v3/role_assignments?user.id=b8ec0b1e5f4d434bb725297f3f867ba7&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=bb34a44869834493b10a99cc751ab6c2 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:50.140642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0e6e7-57b0-4e2a-a20a-56a4972ece6a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:50.299855 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0e6e7-57b0-4e2a-a20a-56a4972ece6a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:50.349837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519193eb-9cfc-4858-b377-b15186b9c4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:50.350482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519193eb-9cfc-4858-b377-b15186b9c4fb None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:50.350771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519193eb-9cfc-4858-b377-b15186b9c4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:50.351044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519193eb-9cfc-4858-b377-b15186b9c4fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:50.363260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519193eb-9cfc-4858-b377-b15186b9c4fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:50.364769 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519193eb-9cfc-4858-b377-b15186b9c4fb None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-519193eb-9cfc-4858-b377-b15186b9c4fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:50.382595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:50.383596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:50.384526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:50.384526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:50.384526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:50.384526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 340/698] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:50.390418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594461e1-de99-4874-baba-224a76231a0b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:50.480602 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:50.496151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8dd7a-7a4e-4451-ba38-4c8f33186031 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:50.500292 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8dd7a-7a4e-4451-ba38-4c8f33186031 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:50.500292 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8dd7a-7a4e-4451-ba38-4c8f33186031 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:50.501387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 341/699] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:50.508358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b2d41-f18d-442a-9e90-dbe65e487473 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:50.510691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b2d41-f18d-442a-9e90-dbe65e487473 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:50.511099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b2d41-f18d-442a-9e90-dbe65e487473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:50.528941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594461e1-de99-4874-baba-224a76231a0b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:50.537600 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-f91b2d41-f18d-442a-9e90-dbe65e487473 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:50.539573 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-f91b2d41-f18d-442a-9e90-dbe65e487473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:50.574793 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb512202-00d9-4fe3-9881-f1bfbe879adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:50.575804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb512202-00d9-4fe3-9881-f1bfbe879adf None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:50.576286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb512202-00d9-4fe3-9881-f1bfbe879adf None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:50.576752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb512202-00d9-4fe3-9881-f1bfbe879adf None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:50.577607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb512202-00d9-4fe3-9881-f1bfbe879adf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:50.578076 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb512202-00d9-4fe3-9881-f1bfbe879adf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:50.579970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb512202-00d9-4fe3-9881-f1bfbe879adf None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:50.600711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 359/700] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:50.603574 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb512202-00d9-4fe3-9881-f1bfbe879adf None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:50.604606 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:50.640577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 342/701] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:50.738872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb512202-00d9-4fe3-9881-f1bfbe879adf None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:50.779647 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:50.786534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1716e49-283d-46fd-af6b-640026a98b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:50.787367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1716e49-283d-46fd-af6b-640026a98b73 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:50.788032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1716e49-283d-46fd-af6b-640026a98b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:50.788310 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1716e49-283d-46fd-af6b-640026a98b73 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:50.793361 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1716e49-283d-46fd-af6b-640026a98b73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:50.795025 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1716e49-283d-46fd-af6b-640026a98b73 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:50.874768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 360/702] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:28:50 2022] POST /identity/v3/users => generated 254 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:28:51.050958 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b2d41-f18d-442a-9e90-dbe65e487473 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:51.194664 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b2d41-f18d-442a-9e90-dbe65e487473 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:51.243781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4c7c8-81eb-482c-8cfb-cc8e9891e3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:51.244430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4c7c8-81eb-482c-8cfb-cc8e9891e3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:51.244730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4c7c8-81eb-482c-8cfb-cc8e9891e3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:51.245002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4c7c8-81eb-482c-8cfb-cc8e9891e3e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:51.259472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4c7c8-81eb-482c-8cfb-cc8e9891e3e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-dfe4c7c8-81eb-482c-8cfb-cc8e9891e3e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:51.265035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:51.266538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:51.268016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:51.268016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:51.268016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:51.268016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:51.268016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:51.268016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:51.268016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:51.268016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:51.268016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:51.268016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:51.268016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:51.268016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:51.268016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 343/703] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:28:51.272456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1716e49-283d-46fd-af6b-640026a98b73 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:51.458393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1716e49-283d-46fd-af6b-640026a98b73 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:51.502803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375e8d9-ae32-4868-9d33-8222ec6f9e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:51.503432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375e8d9-ae32-4868-9d33-8222ec6f9e1f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:51.503712 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375e8d9-ae32-4868-9d33-8222ec6f9e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:51.503985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375e8d9-ae32-4868-9d33-8222ec6f9e1f None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:51.504707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375e8d9-ae32-4868-9d33-8222ec6f9e1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:51.505010 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375e8d9-ae32-4868-9d33-8222ec6f9e1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:51.506406 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375e8d9-ae32-4868-9d33-8222ec6f9e1f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:51.520255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 361/704] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:51.525480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4c7c8-81eb-482c-8cfb-cc8e9891e3e4 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:51.597023 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:51.696466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4c7c8-81eb-482c-8cfb-cc8e9891e3e4 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:51.737729 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca517ba-cdc1-4749-b8dc-4a36a5f42fe4 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:51.737922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca517ba-cdc1-4749-b8dc-4a36a5f42fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:51.738330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca517ba-cdc1-4749-b8dc-4a36a5f42fe4 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:51.739830 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 362/705] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:51.745896 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f5f43-a35d-4827-a3a6-91814b1cf323 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:51.746241 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f5f43-a35d-4827-a3a6-91814b1cf323 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:51.747078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f5f43-a35d-4827-a3a6-91814b1cf323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:51.780976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e04f5f43-a35d-4827-a3a6-91814b1cf323 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:51.782154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45614f8a-0ec0-4241-87f6-d033de44a509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:51.782290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e04f5f43-a35d-4827-a3a6-91814b1cf323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:51.782821 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45614f8a-0ec0-4241-87f6-d033de44a509 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:51.783104 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45614f8a-0ec0-4241-87f6-d033de44a509 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:51.783385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45614f8a-0ec0-4241-87f6-d033de44a509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:51.807714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45614f8a-0ec0-4241-87f6-d033de44a509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:51.809904 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45614f8a-0ec0-4241-87f6-d033de44a509 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-45614f8a-0ec0-4241-87f6-d033de44a509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:51.849956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:51.850983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:51.851882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:51.851882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:51.851882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:51.858942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 344/706] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:51.860817 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:51.870162 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45614f8a-0ec0-4241-87f6-d033de44a509 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:51.915374 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 363/707] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:52.027245 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:52.038607 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45614f8a-0ec0-4241-87f6-d033de44a509 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:52.088872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c52013-325d-41e0-9782-2e3307849b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:52.089535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c52013-325d-41e0-9782-2e3307849b6a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:52.089794 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c52013-325d-41e0-9782-2e3307849b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:52.090065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c52013-325d-41e0-9782-2e3307849b6a None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:52.090731 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c52013-325d-41e0-9782-2e3307849b6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:52.091034 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c52013-325d-41e0-9782-2e3307849b6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:52.092472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c52013-325d-41e0-9782-2e3307849b6a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:52.106361 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 345/708] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:52.111879 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f5f43-a35d-4827-a3a6-91814b1cf323 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:52.223463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f5f43-a35d-4827-a3a6-91814b1cf323 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:52.262509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cee96-e07f-4a84-bf29-5b19ceca323c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:52.263173 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cee96-e07f-4a84-bf29-5b19ceca323c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:52.263457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cee96-e07f-4a84-bf29-5b19ceca323c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:52.263724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cee96-e07f-4a84-bf29-5b19ceca323c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:52.272259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7cee96-e07f-4a84-bf29-5b19ceca323c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:52.273741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7cee96-e07f-4a84-bf29-5b19ceca323c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-1a7cee96-e07f-4a84-bf29-5b19ceca323c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:52.286189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:52.287190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:52.288892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:52.288892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:52.288892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:52.288892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:52.288892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:52.288892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:52.288892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:52.288892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:52.288892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:28:52.288892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:52.288892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 364/709] 10.222.0.70 () {56 vars in 1093 bytes} [Wed Aug 24 16:28:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:52.293713 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c52013-325d-41e0-9782-2e3307849b6a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:52.394804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9efc2-e4aa-4cb3-991d-5c2026d42d7f None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:52.395204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9efc2-e4aa-4cb3-991d-5c2026d42d7f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:52.395591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9efc2-e4aa-4cb3-991d-5c2026d42d7f None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:52.397946 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 365/710] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:52.403717 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e2fea-291e-44f2-9e00-143462bc6b22 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:52.404013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e2fea-291e-44f2-9e00-143462bc6b22 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:52.404349 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e2fea-291e-44f2-9e00-143462bc6b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:52.429565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c52013-325d-41e0-9782-2e3307849b6a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:52.429761 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b40e2fea-291e-44f2-9e00-143462bc6b22 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:52.430786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b40e2fea-291e-44f2-9e00-143462bc6b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:52.469800 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c475b14-1970-41e6-ad8d-81ce7b0d840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:52.470835 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c475b14-1970-41e6-ad8d-81ce7b0d840e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:52.471324 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c475b14-1970-41e6-ad8d-81ce7b0d840e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:52.471789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c475b14-1970-41e6-ad8d-81ce7b0d840e None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:52.473235 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c475b14-1970-41e6-ad8d-81ce7b0d840e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:52.473782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c475b14-1970-41e6-ad8d-81ce7b0d840e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:52.476716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c475b14-1970-41e6-ad8d-81ce7b0d840e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:52.481447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:52.516269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 346/711] 10.222.0.70 () {56 vars in 1133 bytes} [Wed Aug 24 16:28:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:52.520724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c475b14-1970-41e6-ad8d-81ce7b0d840e None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:52.522172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 366/712] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:52.523741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e2fea-291e-44f2-9e00-143462bc6b22 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:52.543843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:52.667596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c475b14-1970-41e6-ad8d-81ce7b0d840e None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:52.675756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e2fea-291e-44f2-9e00-143462bc6b22 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:52.727912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f293b-eb0a-47f5-8743-f8016f6c827c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:52.728545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f293b-eb0a-47f5-8743-f8016f6c827c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:52.728920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f293b-eb0a-47f5-8743-f8016f6c827c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:52.729123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f293b-eb0a-47f5-8743-f8016f6c827c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:52.735143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e65b2-6782-4502-8c73-1cf65c28d4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:52.736115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e65b2-6782-4502-8c73-1cf65c28d4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:52.736637 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e65b2-6782-4502-8c73-1cf65c28d4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:52.737115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e65b2-6782-4502-8c73-1cf65c28d4d7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:52.737917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e65b2-6782-4502-8c73-1cf65c28d4d7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:52.739986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e65b2-6782-4502-8c73-1cf65c28d4d7 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-887e65b2-6782-4502-8c73-1cf65c28d4d7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:52.751881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:52.753212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:52.754058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:52.754058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:52.754058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:52.754058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:52.754058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:52.754058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:52.754058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:52.754058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:52.754058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:52.754058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:52.754058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:28:52.754058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:28:52.754058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:28:52.754058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:52.757622 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 347/713] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:52 2022] GET /identity/v3/services/network => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:52.769919 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e65b2-6782-4502-8c73-1cf65c28d4d7 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:52.773781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:52.821034 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f293b-eb0a-47f5-8743-f8016f6c827c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:52.822710 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f293b-eb0a-47f5-8743-f8016f6c827c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:52.952191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 367/714] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:52.959179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f293b-eb0a-47f5-8743-f8016f6c827c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:52.977075 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:53.008219 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e65b2-6782-4502-8c73-1cf65c28d4d7 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:53.043220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:53.064439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b6274-c212-4a7c-a99d-fd5f8369b63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:53.065451 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b6274-c212-4a7c-a99d-fd5f8369b63b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:53.065794 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b6274-c212-4a7c-a99d-fd5f8369b63b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:53.066465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b6274-c212-4a7c-a99d-fd5f8369b63b None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:53.067331 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b6274-c212-4a7c-a99d-fd5f8369b63b None admin] RBAC: Adding query filter params (name=network) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:53.071620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b6274-c212-4a7c-a99d-fd5f8369b63b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:53.071620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b6274-c212-4a7c-a99d-fd5f8369b63b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:53.079669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 348/715] 10.222.0.70 () {56 vars in 1112 bytes} [Wed Aug 24 16:28:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:53.082144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b6274-c212-4a7c-a99d-fd5f8369b63b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:53.083854 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f293b-eb0a-47f5-8743-f8016f6c827c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:53.128087 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b60f01-adc1-4354-8d57-c65eb9b941ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:53.128935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b60f01-adc1-4354-8d57-c65eb9b941ea None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:53.129290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b60f01-adc1-4354-8d57-c65eb9b941ea None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:53.129632 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b60f01-adc1-4354-8d57-c65eb9b941ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:53.142608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b60f01-adc1-4354-8d57-c65eb9b941ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:53.144075 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b60f01-adc1-4354-8d57-c65eb9b941ea None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-d3b60f01-adc1-4354-8d57-c65eb9b941ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:53.160319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:53.161423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:53.162329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:53.162329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:53.162329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:53.162329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 368/716] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:28:53.162924 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71df61c-660e-4dcb-a992-65f907e645ce None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:53.162924 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71df61c-660e-4dcb-a992-65f907e645ce None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:53.163126 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71df61c-660e-4dcb-a992-65f907e645ce None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:53.168447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 369/717] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:53.168447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71df61c-660e-4dcb-a992-65f907e645ce None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:53.213458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b6274-c212-4a7c-a99d-fd5f8369b63b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:53.252544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b10da-b510-4041-83ac-3df8b8bd81e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:53.253319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b10da-b510-4041-83ac-3df8b8bd81e9 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:53.253639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b10da-b510-4041-83ac-3df8b8bd81e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:53.253920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b10da-b510-4041-83ac-3df8b8bd81e9 None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:53.254650 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b10da-b510-4041-83ac-3df8b8bd81e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:53.256489 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b10da-b510-4041-83ac-3df8b8bd81e9 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:53.292994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71df61c-660e-4dcb-a992-65f907e645ce None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:53.316087 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:53.338539 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d117e39-b6b3-44b2-b2a7-c9d7e3098f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:53.339561 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d117e39-b6b3-44b2-b2a7-c9d7e3098f07 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:53.340660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d117e39-b6b3-44b2-b2a7-c9d7e3098f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:53.341182 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d117e39-b6b3-44b2-b2a7-c9d7e3098f07 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:53.342400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d117e39-b6b3-44b2-b2a7-c9d7e3098f07 None admin] RBAC: Adding query filter params (type=network) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:53.342952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d117e39-b6b3-44b2-b2a7-c9d7e3098f07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:53.345021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d117e39-b6b3-44b2-b2a7-c9d7e3098f07 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:53.351536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 370/718] 10.222.0.70 () {56 vars in 1112 bytes} [Wed Aug 24 16:28:53 2022] GET /identity/v3/services?type=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:53.352413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d117e39-b6b3-44b2-b2a7-c9d7e3098f07 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:53.360899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:53.377183 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 349/719] 10.222.0.70 () {60 vars in 1141 bytes} [Wed Aug 24 16:28:53 2022] POST /identity/v3/services => generated 251 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:28:53.377183 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5ab66-2e34-41d1-ac82-28fec36b3dfb None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:53.377183 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5ab66-2e34-41d1-ac82-28fec36b3dfb None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:53.377183 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5ab66-2e34-41d1-ac82-28fec36b3dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:53.438221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-d9c5ab66-2e34-41d1-ac82-28fec36b3dfb None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:53.440267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d9c5ab66-2e34-41d1-ac82-28fec36b3dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:53.622554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d117e39-b6b3-44b2-b2a7-c9d7e3098f07 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:53.636373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 350/720] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:53 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:53.656283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e9e10-471c-41c1-b5ef-e366e226a58e None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:53.656624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e9e10-471c-41c1-b5ef-e366e226a58e None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:53.656905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e9e10-471c-41c1-b5ef-e366e226a58e None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:53.658565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 351/721] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:28:53.664035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8fab8-9048-4d4e-a6f4-c55d05d62dd8 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:53.664323 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8fab8-9048-4d4e-a6f4-c55d05d62dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:53.664629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8fab8-9048-4d4e-a6f4-c55d05d62dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:53.665982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669815a2-ea27-4537-86fb-466a6874cca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:53.666599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669815a2-ea27-4537-86fb-466a6874cca4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:53.666883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669815a2-ea27-4537-86fb-466a6874cca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:53.667162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669815a2-ea27-4537-86fb-466a6874cca4 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:53.667815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669815a2-ea27-4537-86fb-466a6874cca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:53.668166 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669815a2-ea27-4537-86fb-466a6874cca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:53.669586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669815a2-ea27-4537-86fb-466a6874cca4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:53.682826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 371/722] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:53.684147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-63c8fab8-9048-4d4e-a6f4-c55d05d62dd8 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:53.685520 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-63c8fab8-9048-4d4e-a6f4-c55d05d62dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:53.689638 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669815a2-ea27-4537-86fb-466a6874cca4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:53.743601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:53.770438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 352/723] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:53 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:28:53.806275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669815a2-ea27-4537-86fb-466a6874cca4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:53.844825 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722af51-8163-4d57-9b1f-400a6b1867e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:53.845783 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722af51-8163-4d57-9b1f-400a6b1867e2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:53.846252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722af51-8163-4d57-9b1f-400a6b1867e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:53.846732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722af51-8163-4d57-9b1f-400a6b1867e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:53.854600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4722af51-8163-4d57-9b1f-400a6b1867e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:53.857062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4722af51-8163-4d57-9b1f-400a6b1867e2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:53.869415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-4722af51-8163-4d57-9b1f-400a6b1867e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:53.869415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:53.869415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:53.869415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:53.869415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:53.869415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:53.869415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:53.869415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:53.869415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:53.869415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:53.869415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:53.869415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:53.869415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:53.869415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:53.869415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:53.870417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:53.871922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:53.871922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:53.871922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:53.871922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:53.871922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:53.871922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:53.871922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:53.871922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:53.874311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 372/724] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:53.879736 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8fab8-9048-4d4e-a6f4-c55d05d62dd8 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:53.973750 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:54.030138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8fab8-9048-4d4e-a6f4-c55d05d62dd8 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:54.034484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722af51-8163-4d57-9b1f-400a6b1867e2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:54.085264 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fbe29-ba3b-432c-bda2-25c6d9734f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:54.085894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fbe29-ba3b-432c-bda2-25c6d9734f58 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:54.086174 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fbe29-ba3b-432c-bda2-25c6d9734f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:54.086451 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fbe29-ba3b-432c-bda2-25c6d9734f58 None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:54.087155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fbe29-ba3b-432c-bda2-25c6d9734f58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:54.087459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fbe29-ba3b-432c-bda2-25c6d9734f58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:54.089074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fbe29-ba3b-432c-bda2-25c6d9734f58 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:54.103295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 353/725] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:54.104169 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fbe29-ba3b-432c-bda2-25c6d9734f58 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:54.195768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722af51-8163-4d57-9b1f-400a6b1867e2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:54.199760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:54.241521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fbe29-ba3b-432c-bda2-25c6d9734f58 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:54.248911 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55fb35-bcdb-438c-b9e7-3813a5e0d773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:54.249587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55fb35-bcdb-438c-b9e7-3813a5e0d773 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:54.249853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55fb35-bcdb-438c-b9e7-3813a5e0d773 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:54.250603 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55fb35-bcdb-438c-b9e7-3813a5e0d773 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:54.261388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:54.263540 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55fb35-bcdb-438c-b9e7-3813a5e0d773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-8b55fb35-bcdb-438c-b9e7-3813a5e0d773 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:54.269147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:54.270114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:54.270940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:54.270940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:54.270940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:54.270940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:54.270940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:54.270940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:54.270940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:54.270940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:54.270940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:54.270940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:54.270940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:54.270940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:54.270940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 373/726] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:28:54.271763 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55fb35-bcdb-438c-b9e7-3813a5e0d773 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:54.292703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc249139-6dce-4b77-b0ae-b867c20c86aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:54.294442 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc249139-6dce-4b77-b0ae-b867c20c86aa None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:54.294897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc249139-6dce-4b77-b0ae-b867c20c86aa None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:54.296262 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc249139-6dce-4b77-b0ae-b867c20c86aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:54.311112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc249139-6dce-4b77-b0ae-b867c20c86aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-bc249139-6dce-4b77-b0ae-b867c20c86aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:54.318251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:54.319538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:54.320524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:54.320524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:54.320524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:54.320524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:54.320524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:54.320524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:54.320524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:54.320524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:54.320524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:54.320524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:54.320524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:54.320524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:54.328688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 354/727] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:54.329701 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc249139-6dce-4b77-b0ae-b867c20c86aa None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:54.415280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55fb35-bcdb-438c-b9e7-3813a5e0d773 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:54.462335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54eb17-0482-4d59-967c-f3152c143762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:54.462864 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54eb17-0482-4d59-967c-f3152c143762 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:54.463145 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54eb17-0482-4d59-967c-f3152c143762 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:54.463425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54eb17-0482-4d59-967c-f3152c143762 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:54.464165 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54eb17-0482-4d59-967c-f3152c143762 None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=e333fca7790244f6a4ebceabb55faba4) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:54.464548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54eb17-0482-4d59-967c-f3152c143762 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:54.466579 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54eb17-0482-4d59-967c-f3152c143762 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:54.467994 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc249139-6dce-4b77-b0ae-b867c20c86aa None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:54.478227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 374/728] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:54 2022] GET /identity/v3/role_assignments?user.id=e333fca7790244f6a4ebceabb55faba4&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=e7f48343fc1542979ab4459865109e30 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:54.479811 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54eb17-0482-4d59-967c-f3152c143762 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:54.524913 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afd9bc-5e80-45ca-8442-4245628fe565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:54.526515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afd9bc-5e80-45ca-8442-4245628fe565 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:54.531063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afd9bc-5e80-45ca-8442-4245628fe565 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:54.531770 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afd9bc-5e80-45ca-8442-4245628fe565 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:54.532935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afd9bc-5e80-45ca-8442-4245628fe565 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:54.533459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afd9bc-5e80-45ca-8442-4245628fe565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:54.535216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afd9bc-5e80-45ca-8442-4245628fe565 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:54.553017 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:54.556801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 355/729] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:54.565648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afd9bc-5e80-45ca-8442-4245628fe565 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:54.616828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:54.701607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54eb17-0482-4d59-967c-f3152c143762 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:54.705556 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afd9bc-5e80-45ca-8442-4245628fe565 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:54.752332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a971de-f8f2-421d-9bcc-633df07c547a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:54.752925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a971de-f8f2-421d-9bcc-633df07c547a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:54.753202 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a971de-f8f2-421d-9bcc-633df07c547a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:54.753472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a971de-f8f2-421d-9bcc-633df07c547a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:54.758596 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e6adb-b9ee-402f-a258-a581b5cb2be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:54.759220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e6adb-b9ee-402f-a258-a581b5cb2be3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:54.759498 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e6adb-b9ee-402f-a258-a581b5cb2be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:54.759769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e6adb-b9ee-402f-a258-a581b5cb2be3 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:54.760471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e6adb-b9ee-402f-a258-a581b5cb2be3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:54.760781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e6adb-b9ee-402f-a258-a581b5cb2be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:54.762708 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e6adb-b9ee-402f-a258-a581b5cb2be3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:54.778692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a971de-f8f2-421d-9bcc-633df07c547a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:54.779552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 375/730] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:54.781068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a971de-f8f2-421d-9bcc-633df07c547a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:54.785871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e6adb-b9ee-402f-a258-a581b5cb2be3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-b6a971de-f8f2-421d-9bcc-633df07c547a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:54.800514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:54.802320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:54.803957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:54.803957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:54.803957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:54.806555 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 356/731] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:54.815949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a971de-f8f2-421d-9bcc-633df07c547a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:54.918495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:54.937789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e6adb-b9ee-402f-a258-a581b5cb2be3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:54.988601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174bd2b-b606-4b2c-b5db-0881fead4241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:54.989251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174bd2b-b606-4b2c-b5db-0881fead4241 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:54.989527 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174bd2b-b606-4b2c-b5db-0881fead4241 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:54.989796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174bd2b-b606-4b2c-b5db-0881fead4241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:55.014655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174bd2b-b606-4b2c-b5db-0881fead4241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:55.016605 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174bd2b-b606-4b2c-b5db-0881fead4241 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:55.025802 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-3174bd2b-b606-4b2c-b5db-0881fead4241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:55.045110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:55.046169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:55.047142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:55.047142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:55.047142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:55.051208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 376/732] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:55.052201 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174bd2b-b606-4b2c-b5db-0881fead4241 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:55.053301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a971de-f8f2-421d-9bcc-633df07c547a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:55.118182 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fe859-a7d5-4b9a-84e7-817776ba0c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:55.119185 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fe859-a7d5-4b9a-84e7-817776ba0c7a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:55.119617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fe859-a7d5-4b9a-84e7-817776ba0c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:55.120038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fe859-a7d5-4b9a-84e7-817776ba0c7a None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:55.120906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fe859-a7d5-4b9a-84e7-817776ba0c7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:55.121984 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fe859-a7d5-4b9a-84e7-817776ba0c7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:55.123526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fe859-a7d5-4b9a-84e7-817776ba0c7a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:55.145859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 357/733] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:55.151337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fe859-a7d5-4b9a-84e7-817776ba0c7a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:55.233792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174bd2b-b606-4b2c-b5db-0881fead4241 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:55.310737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802e244-e8bc-4fc7-84ab-0a1a89504fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:55.311278 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802e244-e8bc-4fc7-84ab-0a1a89504fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:55.311560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802e244-e8bc-4fc7-84ab-0a1a89504fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:55.311832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802e244-e8bc-4fc7-84ab-0a1a89504fb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:55.312515 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802e244-e8bc-4fc7-84ab-0a1a89504fb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:55.312821 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802e244-e8bc-4fc7-84ab-0a1a89504fb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:55.314264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802e244-e8bc-4fc7-84ab-0a1a89504fb7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:55.329913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 377/734] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:55.336309 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802e244-e8bc-4fc7-84ab-0a1a89504fb7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:55.340436 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:55.343044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fe859-a7d5-4b9a-84e7-817776ba0c7a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:55.420601 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720be39-7fa8-4516-a0e3-18f20aeb38ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:55.421679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720be39-7fa8-4516-a0e3-18f20aeb38ab None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:55.422169 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720be39-7fa8-4516-a0e3-18f20aeb38ab None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:55.422648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720be39-7fa8-4516-a0e3-18f20aeb38ab None admin] PATH_INFO: `/v3/users/swift` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:55.436582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720be39-7fa8-4516-a0e3-18f20aeb38ab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:55.440901 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720be39-7fa8-4516-a0e3-18f20aeb38ab None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-f720be39-7fa8-4516-a0e3-18f20aeb38ab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:55.460312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:55.461376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:55.463008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:55.463008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:55.463008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:55.463008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:55.463008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:55.463008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:55.463008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:55.463008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:55.463008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:28:55.463008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:55.463008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 358/735] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:55.469677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720be39-7fa8-4516-a0e3-18f20aeb38ab None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:55.504593 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:55.578679 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802e244-e8bc-4fc7-84ab-0a1a89504fb7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:55.622431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720be39-7fa8-4516-a0e3-18f20aeb38ab None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:55.666996 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d41b9e-839b-4ff9-98d2-4f054f7f2279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:55.672565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d41b9e-839b-4ff9-98d2-4f054f7f2279 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:55.672565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d41b9e-839b-4ff9-98d2-4f054f7f2279 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:55.672565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d41b9e-839b-4ff9-98d2-4f054f7f2279 None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:55.672565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d41b9e-839b-4ff9-98d2-4f054f7f2279 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:55.672565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d41b9e-839b-4ff9-98d2-4f054f7f2279 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:55.672565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d41b9e-839b-4ff9-98d2-4f054f7f2279 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:55.677795 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e35d4-82d2-4865-85c2-0a9533b784ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:55.678444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e35d4-82d2-4865-85c2-0a9533b784ed None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:55.678716 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e35d4-82d2-4865-85c2-0a9533b784ed None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:55.679145 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e35d4-82d2-4865-85c2-0a9533b784ed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:55.697683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e35d4-82d2-4865-85c2-0a9533b784ed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:55.704780 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e35d4-82d2-4865-85c2-0a9533b784ed None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:55.708359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 359/736] 10.222.0.70 () {56 vars in 1135 bytes} [Wed Aug 24 16:28:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:55.714709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d41b9e-839b-4ff9-98d2-4f054f7f2279 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-0f5e35d4-82d2-4865-85c2-0a9533b784ed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:55.732556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:55.733787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:55.734885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:55.734885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:55.734885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:55.734885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:55.734885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:55.734885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:55.734885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:55.734885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:55.734885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:28:55.734885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:55.738235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 378/737] 10.222.0.70 () {56 vars in 1117 bytes} [Wed Aug 24 16:28:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:28:55.739930 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a690b-b0f0-4062-bf60-c45f8769caed None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:55.740434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a690b-b0f0-4062-bf60-c45f8769caed None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:55.740904 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a690b-b0f0-4062-bf60-c45f8769caed None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:55.743012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 379/738] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:55.747267 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a690b-b0f0-4062-bf60-c45f8769caed None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:55.825281 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:55.834779 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:55.874879 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d41b9e-839b-4ff9-98d2-4f054f7f2279 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:55.881727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a690b-b0f0-4062-bf60-c45f8769caed None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:55.916331 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cd79a-8f5d-44f5-a29c-eb7d5773d72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:55.917267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cd79a-8f5d-44f5-a29c-eb7d5773d72c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:55.917557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cd79a-8f5d-44f5-a29c-eb7d5773d72c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:55.917836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cd79a-8f5d-44f5-a29c-eb7d5773d72c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:55.921405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4743ce9-23b8-4912-8627-17e224214853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:55.922277 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4743ce9-23b8-4912-8627-17e224214853 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:55.922642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4743ce9-23b8-4912-8627-17e224214853 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:55.922927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4743ce9-23b8-4912-8627-17e224214853 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:55.923704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4743ce9-23b8-4912-8627-17e224214853 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:55.924095 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4743ce9-23b8-4912-8627-17e224214853 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:55.925933 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4743ce9-23b8-4912-8627-17e224214853 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:55.958084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 380/739] 10.222.0.70 () {56 vars in 1149 bytes} [Wed Aug 24 16:28:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:28:55.959731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a775963-9ea4-41a8-a1cc-aa1f71587f8e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:55.960100 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a775963-9ea4-41a8-a1cc-aa1f71587f8e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:55.960390 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a775963-9ea4-41a8-a1cc-aa1f71587f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:55.978994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-6a775963-9ea4-41a8-a1cc-aa1f71587f8e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:55.983687 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6a775963-9ea4-41a8-a1cc-aa1f71587f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:56.019443 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cd79a-8f5d-44f5-a29c-eb7d5773d72c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:56.022002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cd79a-8f5d-44f5-a29c-eb7d5773d72c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:56.062636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 381/740] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:56.063730 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa027e6-22b4-4e2f-8da9-5c3d4d1243c5 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:56.064014 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa027e6-22b4-4e2f-8da9-5c3d4d1243c5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:56.064285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa027e6-22b4-4e2f-8da9-5c3d4d1243c5 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:56.068598 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 382/741] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:56.068598 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa027e6-22b4-4e2f-8da9-5c3d4d1243c5 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:56.113973 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:56.129324 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:56.167311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 360/742] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:55 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:56.169857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16634113-cb67-4a8f-bbde-08232fc0dd7a None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:56.170258 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16634113-cb67-4a8f-bbde-08232fc0dd7a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:56.170643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16634113-cb67-4a8f-bbde-08232fc0dd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:56.190290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa027e6-22b4-4e2f-8da9-5c3d4d1243c5 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:56.196295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-16634113-cb67-4a8f-bbde-08232fc0dd7a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:56.197687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-16634113-cb67-4a8f-bbde-08232fc0dd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:56.231924 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5e263-a8c0-4e92-8851-dc8917f6f82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:56.232996 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5e263-a8c0-4e92-8851-dc8917f6f82e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:56.233518 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5e263-a8c0-4e92-8851-dc8917f6f82e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:56.233976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5e263-a8c0-4e92-8851-dc8917f6f82e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:56.303351 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 361/743] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:28:56.305532 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16634113-cb67-4a8f-bbde-08232fc0dd7a None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:56.335680 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5e263-a8c0-4e92-8851-dc8917f6f82e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:56.337519 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5e263-a8c0-4e92-8851-dc8917f6f82e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:56.409197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:56.461777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:56.468998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 383/744] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:56 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:28:56.469898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5e263-a8c0-4e92-8851-dc8917f6f82e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:56.471561 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16634113-cb67-4a8f-bbde-08232fc0dd7a None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:56.524595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb759b-c08a-424c-b5cd-cec3249a995f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:56.524940 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb759b-c08a-424c-b5cd-cec3249a995f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:56.525379 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb759b-c08a-424c-b5cd-cec3249a995f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:56.525747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb759b-c08a-424c-b5cd-cec3249a995f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:56.540071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb759b-c08a-424c-b5cd-cec3249a995f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:56.542309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb759b-c08a-424c-b5cd-cec3249a995f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-0beb759b-c08a-424c-b5cd-cec3249a995f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:56.566892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:56.568171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:56.569315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:56.569315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:56.569315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:56.570671 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 362/745] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:56.571679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb759b-c08a-424c-b5cd-cec3249a995f None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:56.617391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5e263-a8c0-4e92-8851-dc8917f6f82e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:56.656511 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b28ed-7f0d-4412-8f9d-1f5b6824ad8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:56.657383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b28ed-7f0d-4412-8f9d-1f5b6824ad8f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:56.657761 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b28ed-7f0d-4412-8f9d-1f5b6824ad8f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:56.658125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b28ed-7f0d-4412-8f9d-1f5b6824ad8f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:56.658842 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b28ed-7f0d-4412-8f9d-1f5b6824ad8f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:56.668518 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b28ed-7f0d-4412-8f9d-1f5b6824ad8f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-d02b28ed-7f0d-4412-8f9d-1f5b6824ad8f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:56.673798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:56.675083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:56.676041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:56.676041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:56.676041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:56.676041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:56.676041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:56.676041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:56.676041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:56.676041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:56.676041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:56.676041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:56.676041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:28:56.676041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:28:56.676041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:28:56.676041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:56.681704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 384/746] 10.222.0.70 () {56 vars in 1129 bytes} [Wed Aug 24 16:28:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:56.682937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b28ed-7f0d-4412-8f9d-1f5b6824ad8f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:56.701131 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:56.702517 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb759b-c08a-424c-b5cd-cec3249a995f None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:56.744943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae276e6-d281-4d41-bf16-28f5e1579b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:56.745991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae276e6-d281-4d41-bf16-28f5e1579b21 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:56.746349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae276e6-d281-4d41-bf16-28f5e1579b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:56.746718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae276e6-d281-4d41-bf16-28f5e1579b21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:56.759670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae276e6-d281-4d41-bf16-28f5e1579b21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:56.761212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae276e6-d281-4d41-bf16-28f5e1579b21 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:56.772556 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-bae276e6-d281-4d41-bf16-28f5e1579b21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:56.779884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:56.788986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:56.789859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:56.789859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:56.789859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:56.789859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 363/747] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:56.789859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae276e6-d281-4d41-bf16-28f5e1579b21 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:56.822718 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b28ed-7f0d-4412-8f9d-1f5b6824ad8f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:56.865059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d85d7-0f9b-4694-a917-68758558cfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:56.865673 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d85d7-0f9b-4694-a917-68758558cfc0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:56.865954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d85d7-0f9b-4694-a917-68758558cfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:56.866231 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d85d7-0f9b-4694-a917-68758558cfc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:56.866883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d85d7-0f9b-4694-a917-68758558cfc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:56.867188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d85d7-0f9b-4694-a917-68758558cfc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:56.868628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d85d7-0f9b-4694-a917-68758558cfc0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:56.881732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 385/748] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:56.887411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d85d7-0f9b-4694-a917-68758558cfc0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:56.916025 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae276e6-d281-4d41-bf16-28f5e1579b21 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:56.958366 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1d3a7-08f7-4988-89b2-81f376a9f704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:56.959260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1d3a7-08f7-4988-89b2-81f376a9f704 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:56.959720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1d3a7-08f7-4988-89b2-81f376a9f704 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:56.960170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1d3a7-08f7-4988-89b2-81f376a9f704 None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:56.961225 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1d3a7-08f7-4988-89b2-81f376a9f704 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:56.961726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1d3a7-08f7-4988-89b2-81f376a9f704 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:56.964161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1d3a7-08f7-4988-89b2-81f376a9f704 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:56.970636 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 364/749] 10.222.0.70 () {56 vars in 1124 bytes} [Wed Aug 24 16:28:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:56.971522 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1d3a7-08f7-4988-89b2-81f376a9f704 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:56.983304 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:57.015524 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d85d7-0f9b-4694-a917-68758558cfc0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:57.044014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:57.068828 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021dba2-b9d9-44ba-9f11-2d34e0d494f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:57.069582 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021dba2-b9d9-44ba-9f11-2d34e0d494f9 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:57.069900 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021dba2-b9d9-44ba-9f11-2d34e0d494f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:57.070204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021dba2-b9d9-44ba-9f11-2d34e0d494f9 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:57.071078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021dba2-b9d9-44ba-9f11-2d34e0d494f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:57.073206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021dba2-b9d9-44ba-9f11-2d34e0d494f9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:57.094310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1d3a7-08f7-4988-89b2-81f376a9f704 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:57.143558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 386/750] 10.222.0.70 () {60 vars in 1141 bytes} [Wed Aug 24 16:28:56 2022] POST /identity/v3/services => generated 240 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:28:57.145160 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef99602-140f-4484-983a-6385a2b7400c None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:57.145597 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef99602-140f-4484-983a-6385a2b7400c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:57.146082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef99602-140f-4484-983a-6385a2b7400c None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:57.147647 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 387/751] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:57.148495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef99602-140f-4484-983a-6385a2b7400c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:57.159424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067832db-73c8-45bd-a504-405cdbcaf8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:57.160272 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067832db-73c8-45bd-a504-405cdbcaf8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:57.160687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067832db-73c8-45bd-a504-405cdbcaf8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:57.161076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067832db-73c8-45bd-a504-405cdbcaf8f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:57.161868 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067832db-73c8-45bd-a504-405cdbcaf8f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:57.162285 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067832db-73c8-45bd-a504-405cdbcaf8f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:57.164002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067832db-73c8-45bd-a504-405cdbcaf8f1 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:57.180510 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 365/752] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:57.181773 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067832db-73c8-45bd-a504-405cdbcaf8f1 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:57.293708 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef99602-140f-4484-983a-6385a2b7400c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:57.319248 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067832db-73c8-45bd-a504-405cdbcaf8f1 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:57.335265 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96604bb-2bcc-467a-895f-3daf99c6e6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:57.335920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96604bb-2bcc-467a-895f-3daf99c6e6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:57.336201 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96604bb-2bcc-467a-895f-3daf99c6e6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:57.337022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96604bb-2bcc-467a-895f-3daf99c6e6a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:57.339386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:57.347094 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96604bb-2bcc-467a-895f-3daf99c6e6a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:57.348181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:57.348846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96604bb-2bcc-467a-895f-3daf99c6e6a4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:57.360971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-c96604bb-2bcc-467a-895f-3daf99c6e6a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:57.360971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:57.360971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:57.360971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:57.360971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:57.360971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:57.360971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:57.360971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:57.360971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:57.360971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:57.360971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:57.360971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:57.360971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:57.360971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:57.360971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:57.362945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:57.364056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:57.364056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:57.364056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:57.364056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:57.364056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:57.364056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:57.364056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:57.364056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:57.364056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 388/753] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:57.364056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83345d33-de94-40dc-ad53-a7384bfb0733 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:57.364056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83345d33-de94-40dc-ad53-a7384bfb0733 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:57.364720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83345d33-de94-40dc-ad53-a7384bfb0733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:57.375145 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11155828-f9e1-4750-9d80-1f8a55285ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:57.375145 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11155828-f9e1-4750-9d80-1f8a55285ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:57.375145 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11155828-f9e1-4750-9d80-1f8a55285ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:57.375145 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11155828-f9e1-4750-9d80-1f8a55285ff1 None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:57.375624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11155828-f9e1-4750-9d80-1f8a55285ff1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:57.375723 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11155828-f9e1-4750-9d80-1f8a55285ff1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:57.377224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11155828-f9e1-4750-9d80-1f8a55285ff1 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:57.382703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 366/754] 10.222.0.70 () {56 vars in 1124 bytes} [Wed Aug 24 16:28:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:57.383553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11155828-f9e1-4750-9d80-1f8a55285ff1 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:57.387525 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-83345d33-de94-40dc-ad53-a7384bfb0733 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:57.389138 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-83345d33-de94-40dc-ad53-a7384bfb0733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:57.478520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 389/755] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:57.480892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83345d33-de94-40dc-ad53-a7384bfb0733 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:57.514306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11155828-f9e1-4750-9d80-1f8a55285ff1 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:57.574099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f191c-aaef-4d9b-8bcd-00fa8dfa0df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:57.575758 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f191c-aaef-4d9b-8bcd-00fa8dfa0df5 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:57.576209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f191c-aaef-4d9b-8bcd-00fa8dfa0df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:57.576651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f191c-aaef-4d9b-8bcd-00fa8dfa0df5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:57.587856 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f191c-aaef-4d9b-8bcd-00fa8dfa0df5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:57.590387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f191c-aaef-4d9b-8bcd-00fa8dfa0df5 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:57.608249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83345d33-de94-40dc-ad53-a7384bfb0733 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:57.608806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-849f191c-aaef-4d9b-8bcd-00fa8dfa0df5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:57.608806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:57.608806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:57.608806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:57.608806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:57.608806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:57.608806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:57.608806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:57.608806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:57.608806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:57.608806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:57.608806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:57.608806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:57.608806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:57.608806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:57.609912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:57.611003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:57.611003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:57.611003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:57.611003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:57.611003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:57.611003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:57.611003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:57.611003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:28:57.619078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 367/756] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:57.619078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f191c-aaef-4d9b-8bcd-00fa8dfa0df5 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:57.645776 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:57.663686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34a1fe-76d4-47db-a04b-0aac283a8bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:57.664865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34a1fe-76d4-47db-a04b-0aac283a8bdc None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:57.665296 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34a1fe-76d4-47db-a04b-0aac283a8bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:57.665715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34a1fe-76d4-47db-a04b-0aac283a8bdc None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:57.666601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34a1fe-76d4-47db-a04b-0aac283a8bdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:57.667308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34a1fe-76d4-47db-a04b-0aac283a8bdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:57.671182 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34a1fe-76d4-47db-a04b-0aac283a8bdc None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:57.692253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 390/757] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:57.694104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34a1fe-76d4-47db-a04b-0aac283a8bdc None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:57.699923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:57.801464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f191c-aaef-4d9b-8bcd-00fa8dfa0df5 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:57.854553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d0df9-fd5c-480d-a567-c8416c18c18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:57.855176 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d0df9-fd5c-480d-a567-c8416c18c18b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:57.856019 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d0df9-fd5c-480d-a567-c8416c18c18b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:57.856293 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d0df9-fd5c-480d-a567-c8416c18c18b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:57.857070 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d0df9-fd5c-480d-a567-c8416c18c18b None admin] RBAC: Adding query filter params (interface=public, service_id=67b88a0afaec4a629999fdc3bb0d293c, region_id=RegionOne) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:57.857382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d0df9-fd5c-480d-a567-c8416c18c18b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:57.858797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d0df9-fd5c-480d-a567-c8416c18c18b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:57.860137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34a1fe-76d4-47db-a04b-0aac283a8bdc None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:57.873956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 368/758] 10.222.0.70 () {56 vars in 1251 bytes} [Wed Aug 24 16:28:57 2022] GET /identity/v3/endpoints?service_id=67b88a0afaec4a629999fdc3bb0d293c&interface=public®ion_id=RegionOne => generated 194 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:57.873956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d0df9-fd5c-480d-a567-c8416c18c18b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:57.914667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cb38b-4bd6-4733-b935-93ab98b2dd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:57.915568 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cb38b-4bd6-4733-b935-93ab98b2dd2d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:57.915998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cb38b-4bd6-4733-b935-93ab98b2dd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:57.916439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cb38b-4bd6-4733-b935-93ab98b2dd2d None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:57.917290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cb38b-4bd6-4733-b935-93ab98b2dd2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:57.917773 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cb38b-4bd6-4733-b935-93ab98b2dd2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:57.919956 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cb38b-4bd6-4733-b935-93ab98b2dd2d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:57.934185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 391/759] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:57.935085 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cb38b-4bd6-4733-b935-93ab98b2dd2d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:58.005747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:58.008145 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d0df9-fd5c-480d-a567-c8416c18c18b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:58.032627 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:58.067080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7775a2-f3db-4f81-a360-a9dcfcf9ef4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:58.067815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7775a2-f3db-4f81-a360-a9dcfcf9ef4e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:58.068167 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7775a2-f3db-4f81-a360-a9dcfcf9ef4e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:58.068559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7775a2-f3db-4f81-a360-a9dcfcf9ef4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:58.069373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7775a2-f3db-4f81-a360-a9dcfcf9ef4e None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=b8ec0b1e5f4d434bb725297f3f867ba7) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:58.069759 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7775a2-f3db-4f81-a360-a9dcfcf9ef4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:58.071348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7775a2-f3db-4f81-a360-a9dcfcf9ef4e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:58.078796 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 369/760] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:57 2022] GET /identity/v3/role_assignments?user.id=b8ec0b1e5f4d434bb725297f3f867ba7&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=e7f48343fc1542979ab4459865109e30 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:58.079789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7775a2-f3db-4f81-a360-a9dcfcf9ef4e None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:58.098975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cb38b-4bd6-4733-b935-93ab98b2dd2d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:58.147639 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ebba5-a43b-4c40-924f-bdb8dc9b58e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:58.148533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ebba5-a43b-4c40-924f-bdb8dc9b58e0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:58.148533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ebba5-a43b-4c40-924f-bdb8dc9b58e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:58.148836 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ebba5-a43b-4c40-924f-bdb8dc9b58e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:58.175487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ebba5-a43b-4c40-924f-bdb8dc9b58e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-ee5ebba5-a43b-4c40-924f-bdb8dc9b58e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:58.191386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:58.193291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:58.195023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:58.195023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:58.195023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:58.195023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:58.195023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:58.195023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:58.195023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:58.195023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:58.195023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:58.195023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:58.195023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:58.195023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:58.202572 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 392/761] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:58.204604 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ebba5-a43b-4c40-924f-bdb8dc9b58e0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:58.277112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7775a2-f3db-4f81-a360-a9dcfcf9ef4e None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:58.322752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42edb1-9070-4e52-ab7d-137720f5b97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:58.323386 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42edb1-9070-4e52-ab7d-137720f5b97f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:58.323663 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42edb1-9070-4e52-ab7d-137720f5b97f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:58.323930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42edb1-9070-4e52-ab7d-137720f5b97f None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:58.325144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42edb1-9070-4e52-ab7d-137720f5b97f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:58.326517 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42edb1-9070-4e52-ab7d-137720f5b97f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:58.333934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 370/762] 10.222.0.70 () {56 vars in 1087 bytes} [Wed Aug 24 16:28:58 2022] GET /identity/v3/services => generated 772 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:58.334810 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42edb1-9070-4e52-ab7d-137720f5b97f None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:58.414761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:58.484268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42edb1-9070-4e52-ab7d-137720f5b97f None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:58.494963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ebba5-a43b-4c40-924f-bdb8dc9b58e0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:58.498211 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:58.549006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53898664-5252-44e7-8cb4-26b9f17c98e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:58.549006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53898664-5252-44e7-8cb4-26b9f17c98e6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:58.549006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53898664-5252-44e7-8cb4-26b9f17c98e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:58.549006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53898664-5252-44e7-8cb4-26b9f17c98e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:58.549363 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53898664-5252-44e7-8cb4-26b9f17c98e6 None admin] RBAC: Adding query filter params (role.id=bb34a44869834493b10a99cc751ab6c2, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=7f3899f2eb554f2fb5a7c43f660b51d5) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:58.549363 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53898664-5252-44e7-8cb4-26b9f17c98e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:58.549363 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53898664-5252-44e7-8cb4-26b9f17c98e6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:58.559422 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 393/763] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:58 2022] GET /identity/v3/role_assignments?user.id=7f3899f2eb554f2fb5a7c43f660b51d5&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=bb34a44869834493b10a99cc751ab6c2 => generated 259 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:58.566038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72e355-4872-410e-91b7-f828d425e73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:58.566683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72e355-4872-410e-91b7-f828d425e73f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:58.566968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72e355-4872-410e-91b7-f828d425e73f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:58.567245 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72e355-4872-410e-91b7-f828d425e73f None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:58.567944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72e355-4872-410e-91b7-f828d425e73f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:58.568249 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72e355-4872-410e-91b7-f828d425e73f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:58.569705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72e355-4872-410e-91b7-f828d425e73f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:58.584378 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 371/764] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:58.590961 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53898664-5252-44e7-8cb4-26b9f17c98e6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:58.756792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53898664-5252-44e7-8cb4-26b9f17c98e6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:58.805013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05c523-defe-4902-ae40-24dc7e6e65b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:58.805715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05c523-defe-4902-ae40-24dc7e6e65b7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:58.805991 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05c523-defe-4902-ae40-24dc7e6e65b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:58.806263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05c523-defe-4902-ae40-24dc7e6e65b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:58.823620 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05c523-defe-4902-ae40-24dc7e6e65b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:58.825742 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05c523-defe-4902-ae40-24dc7e6e65b7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-9a05c523-defe-4902-ae40-24dc7e6e65b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:58.844872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:58.846038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:58.846955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:58.846955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:58.846955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:58.849103 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 394/765] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:58.854906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72e355-4872-410e-91b7-f828d425e73f None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:59.029379 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:59.033444 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72e355-4872-410e-91b7-f828d425e73f None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:59.085154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32e8b3-5e4d-47e8-ae5f-28266d8f87e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:59.085789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32e8b3-5e4d-47e8-ae5f-28266d8f87e5 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:59.086165 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32e8b3-5e4d-47e8-ae5f-28266d8f87e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:59.086447 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32e8b3-5e4d-47e8-ae5f-28266d8f87e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:59.087111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32e8b3-5e4d-47e8-ae5f-28266d8f87e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:59.087418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32e8b3-5e4d-47e8-ae5f-28266d8f87e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:59.088977 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32e8b3-5e4d-47e8-ae5f-28266d8f87e5 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:59.106855 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 372/766] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:59.111981 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05c523-defe-4902-ae40-24dc7e6e65b7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:59.148511 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:59.263174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05c523-defe-4902-ae40-24dc7e6e65b7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:59.312907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e1738-3700-491f-ac0d-381b26907aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:59.313909 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e1738-3700-491f-ac0d-381b26907aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:59.314394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e1738-3700-491f-ac0d-381b26907aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:59.314869 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e1738-3700-491f-ac0d-381b26907aa5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:59.327487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e1738-3700-491f-ac0d-381b26907aa5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:59.329121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e1738-3700-491f-ac0d-381b26907aa5 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-eb2e1738-3700-491f-ac0d-381b26907aa5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:59.343112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:59.344125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:59.345144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:59.345144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:59.345144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:59.345144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:59.345144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:59.345144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:59.345144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:59.345144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:59.345144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:28:59.345144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:59.345144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 395/767] 10.222.0.70 () {56 vars in 1093 bytes} [Wed Aug 24 16:28:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:59.353608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32e8b3-5e4d-47e8-ae5f-28266d8f87e5 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:59.499677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32e8b3-5e4d-47e8-ae5f-28266d8f87e5 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:59.547725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382789c-9d03-4278-a5ae-b8cd3ecbe051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:59.548366 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382789c-9d03-4278-a5ae-b8cd3ecbe051 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:59.548685 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382789c-9d03-4278-a5ae-b8cd3ecbe051 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:59.548965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382789c-9d03-4278-a5ae-b8cd3ecbe051 None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:59.549714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382789c-9d03-4278-a5ae-b8cd3ecbe051 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:59.550029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382789c-9d03-4278-a5ae-b8cd3ecbe051 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:59.551497 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382789c-9d03-4278-a5ae-b8cd3ecbe051 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:59.571644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 373/768] 10.222.0.70 () {56 vars in 1133 bytes} [Wed Aug 24 16:28:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:59.581645 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e1738-3700-491f-ac0d-381b26907aa5 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:59.602130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7517d-0897-4c57-825a-9357a026d513 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:59.602449 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7517d-0897-4c57-825a-9357a026d513 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:59.602692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7517d-0897-4c57-825a-9357a026d513 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:59.604349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 374/769] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:59.614121 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc64c4-1025-4d46-a4ca-2f23f0cb91c9 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:59.614412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc64c4-1025-4d46-a4ca-2f23f0cb91c9 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:59.614696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc64c4-1025-4d46-a4ca-2f23f0cb91c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:59.645697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-62fc64c4-1025-4d46-a4ca-2f23f0cb91c9 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:59.646960 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-62fc64c4-1025-4d46-a4ca-2f23f0cb91c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:59.675807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:59.773308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 375/770] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:28:59.785033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:59.806838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e1738-3700-491f-ac0d-381b26907aa5 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:59.856100 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565c26c-3943-47ce-b40f-547238eef155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:59.856940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565c26c-3943-47ce-b40f-547238eef155 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:59.857220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565c26c-3943-47ce-b40f-547238eef155 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:59.857490 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565c26c-3943-47ce-b40f-547238eef155 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:59.877096 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565c26c-3943-47ce-b40f-547238eef155 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:59.878984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565c26c-3943-47ce-b40f-547238eef155 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-e565c26c-3943-47ce-b40f-547238eef155 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:59.901939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:59.903747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:59.904664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:59.904664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:59.904664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:28:59.904664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 396/771] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:59.911109 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc64c4-1025-4d46-a4ca-2f23f0cb91c9 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:00.058719 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc64c4-1025-4d46-a4ca-2f23f0cb91c9 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:00.060976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565c26c-3943-47ce-b40f-547238eef155 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:00.134979 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7aca57-0a7e-4468-8646-42306a9995f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:00.135519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7aca57-0a7e-4468-8646-42306a9995f5 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:00.135793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7aca57-0a7e-4468-8646-42306a9995f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:00.136062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7aca57-0a7e-4468-8646-42306a9995f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:00.137313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7aca57-0a7e-4468-8646-42306a9995f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:00.137643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7aca57-0a7e-4468-8646-42306a9995f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:00.139022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7aca57-0a7e-4468-8646-42306a9995f5 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:00.167127 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 376/772] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:00.171716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7aca57-0a7e-4468-8646-42306a9995f5 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:00.187936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:00.267988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565c26c-3943-47ce-b40f-547238eef155 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:00.331697 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:00.339881 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c44d88-9626-4dda-bfd9-9bb1a720b3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:00.340552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c44d88-9626-4dda-bfd9-9bb1a720b3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:00.340834 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c44d88-9626-4dda-bfd9-9bb1a720b3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:00.341116 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c44d88-9626-4dda-bfd9-9bb1a720b3e4 None admin] PATH_INFO: `/v3/services/network` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:00.341707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c44d88-9626-4dda-bfd9-9bb1a720b3e4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:00.343094 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c44d88-9626-4dda-bfd9-9bb1a720b3e4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f8c44d88-9626-4dda-bfd9-9bb1a720b3e4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:00.365973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:00.367005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:00.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:00.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:00.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:00.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:00.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:00.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:00.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:00.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:00.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:00.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:00.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:00.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:00.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:29:00.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:00.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 397/773] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:00 2022] GET /identity/v3/services/network => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:00.374272 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7aca57-0a7e-4468-8646-42306a9995f5 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:00.377058 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c44d88-9626-4dda-bfd9-9bb1a720b3e4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:00.448006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43b9f1-aa56-4bdb-9e22-96762fb2c681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:00.448696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43b9f1-aa56-4bdb-9e22-96762fb2c681 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:00.449542 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43b9f1-aa56-4bdb-9e22-96762fb2c681 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:00.449823 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43b9f1-aa56-4bdb-9e22-96762fb2c681 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:00.462804 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43b9f1-aa56-4bdb-9e22-96762fb2c681 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:00.464267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43b9f1-aa56-4bdb-9e22-96762fb2c681 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:00.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-ff43b9f1-aa56-4bdb-9e22-96762fb2c681 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:00.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:00.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:00.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:00.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:00.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:00.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:00.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:00.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:00.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:00.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:00.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:00.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:00.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:00.487321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:00.488270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:00.489192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:00.489192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:00.489192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:00.489192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:00.489192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:00.489192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:00.489192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:00.489192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:00.491066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 377/774] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:00.497192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43b9f1-aa56-4bdb-9e22-96762fb2c681 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:00.571892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:00.610571 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43b9f1-aa56-4bdb-9e22-96762fb2c681 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:00.616541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c44d88-9626-4dda-bfd9-9bb1a720b3e4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:00.649268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40146076-dc1d-4e1d-a5e4-64721f08f30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:00.650153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40146076-dc1d-4e1d-a5e4-64721f08f30a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:00.650616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40146076-dc1d-4e1d-a5e4-64721f08f30a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:00.651033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40146076-dc1d-4e1d-a5e4-64721f08f30a None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:00.652154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40146076-dc1d-4e1d-a5e4-64721f08f30a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:00.652499 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40146076-dc1d-4e1d-a5e4-64721f08f30a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:00.653966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40146076-dc1d-4e1d-a5e4-64721f08f30a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:00.661608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649a5dd-36e7-4ce9-9dc3-794e7af4cc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:00.662608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649a5dd-36e7-4ce9-9dc3-794e7af4cc8d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:00.666883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649a5dd-36e7-4ce9-9dc3-794e7af4cc8d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:00.666883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649a5dd-36e7-4ce9-9dc3-794e7af4cc8d None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:00.666883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649a5dd-36e7-4ce9-9dc3-794e7af4cc8d None admin] RBAC: Adding query filter params (name=network) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:00.666883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649a5dd-36e7-4ce9-9dc3-794e7af4cc8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:00.668319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649a5dd-36e7-4ce9-9dc3-794e7af4cc8d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:00.674999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 378/775] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:00.674999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28078f2b-aabd-42af-988a-fda2bcd506a7 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:00.674999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28078f2b-aabd-42af-988a-fda2bcd506a7 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:00.674999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28078f2b-aabd-42af-988a-fda2bcd506a7 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:00.674999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 379/776] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:00.676702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28078f2b-aabd-42af-988a-fda2bcd506a7 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:00.680270 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 398/777] 10.222.0.70 () {56 vars in 1112 bytes} [Wed Aug 24 16:29:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:00.688313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649a5dd-36e7-4ce9-9dc3-794e7af4cc8d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:00.793221 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:00.869747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649a5dd-36e7-4ce9-9dc3-794e7af4cc8d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:00.924851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190e386-2210-41b8-b5bb-a0310580f724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:00.925460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e386-2210-41b8-b5bb-a0310580f724 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:00.925737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e386-2210-41b8-b5bb-a0310580f724 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:00.926006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e386-2210-41b8-b5bb-a0310580f724 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:00.926648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190e386-2210-41b8-b5bb-a0310580f724 None admin] RBAC: Adding query filter params (type=network) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:00.926936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190e386-2210-41b8-b5bb-a0310580f724 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:00.928281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190e386-2210-41b8-b5bb-a0310580f724 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:00.933129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 399/778] 10.222.0.70 () {56 vars in 1112 bytes} [Wed Aug 24 16:29:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:00.934293 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05a002-b366-433b-85c8-f1fe903af02f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:00.934571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05a002-b366-433b-85c8-f1fe903af02f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:00.934848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05a002-b366-433b-85c8-f1fe903af02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:00.952414 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ec05a002-b366-433b-85c8-f1fe903af02f None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:00.953604 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ec05a002-b366-433b-85c8-f1fe903af02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:00.977284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28078f2b-aabd-42af-988a-fda2bcd506a7 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:01.005618 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:01.014409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072dda65-c9f9-45f5-a1d9-c88ae5845ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:01.019250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072dda65-c9f9-45f5-a1d9-c88ae5845ec8 None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.019250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072dda65-c9f9-45f5-a1d9-c88ae5845ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.019250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072dda65-c9f9-45f5-a1d9-c88ae5845ec8 None admin] PATH_INFO: `/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/e333fca7790244f6a4ebceabb55faba4/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.019250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072dda65-c9f9-45f5-a1d9-c88ae5845ec8 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc920dce30734ac7a69dc039d8c74ce1, user_id=e333fca7790244f6a4ebceabb55faba4, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:01.019250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072dda65-c9f9-45f5-a1d9-c88ae5845ec8 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:01.022090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 400/779] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:01.023371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939317c0-165c-4bdc-ac3b-b36bb47b37c8 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.023660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939317c0-165c-4bdc-ac3b-b36bb47b37c8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.023952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939317c0-165c-4bdc-ac3b-b36bb47b37c8 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.025237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 401/780] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:01.029578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed89969-782f-4a5e-ab6a-536c65ceab32 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.030145 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed89969-782f-4a5e-ab6a-536c65ceab32 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.030466 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed89969-782f-4a5e-ab6a-536c65ceab32 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.031665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 402/781] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:01.032377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed89969-782f-4a5e-ab6a-536c65ceab32 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:01.090614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:01.112432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 380/782] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:29:00 2022] PUT /identity/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/e333fca7790244f6a4ebceabb55faba4/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:29:01.112432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb8264-6192-49ec-9003-c2406367e597 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.112432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb8264-6192-49ec-9003-c2406367e597 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.112432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb8264-6192-49ec-9003-c2406367e597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.126893 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-cceb8264-6192-49ec-9003-c2406367e597 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:01.128286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-cceb8264-6192-49ec-9003-c2406367e597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:01.157251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed89969-782f-4a5e-ab6a-536c65ceab32 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:01.212526 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65cfdc-90b3-457e-bba6-a6d3158ec510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:01.213333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65cfdc-90b3-457e-bba6-a6d3158ec510 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.213769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65cfdc-90b3-457e-bba6-a6d3158ec510 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.214047 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 381/783] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:01.214364 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65cfdc-90b3-457e-bba6-a6d3158ec510 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.215617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65cfdc-90b3-457e-bba6-a6d3158ec510 None admin] RBAC: Adding query filter params (interface=public, service_id=ae5e5752d3744146825d1a0891176ece, region_id=RegionOne) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:01.215934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1698a3-233a-477a-ade6-ae9e425fc642 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.216040 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65cfdc-90b3-457e-bba6-a6d3158ec510 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:01.216209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1698a3-233a-477a-ade6-ae9e425fc642 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.216508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1698a3-233a-477a-ade6-ae9e425fc642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.217440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65cfdc-90b3-457e-bba6-a6d3158ec510 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:01.228179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 403/784] 10.222.0.70 () {56 vars in 1251 bytes} [Wed Aug 24 16:29:01 2022] GET /identity/v3/endpoints?service_id=ae5e5752d3744146825d1a0891176ece&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:01.234865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65cfdc-90b3-457e-bba6-a6d3158ec510 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:01.239307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-9c1698a3-233a-477a-ade6-ae9e425fc642 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:01.240603 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-9c1698a3-233a-477a-ade6-ae9e425fc642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:01.330706 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 382/785] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:01.331658 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1698a3-233a-477a-ade6-ae9e425fc642 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:01.397278 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:01.402733 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65cfdc-90b3-457e-bba6-a6d3158ec510 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:01.429721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:01.447502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d48a1f-fdf7-4fc4-a435-cfd66e26a6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:01.448232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d48a1f-fdf7-4fc4-a435-cfd66e26a6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.448708 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d48a1f-fdf7-4fc4-a435-cfd66e26a6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.449353 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d48a1f-fdf7-4fc4-a435-cfd66e26a6c4 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.449921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d48a1f-fdf7-4fc4-a435-cfd66e26a6c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:01.451301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d48a1f-fdf7-4fc4-a435-cfd66e26a6c4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:01.456895 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 404/786] 10.222.0.70 () {56 vars in 1087 bytes} [Wed Aug 24 16:29:01 2022] GET /identity/v3/services => generated 772 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:01.471803 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1698a3-233a-477a-ade6-ae9e425fc642 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:01.520168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a93026-d8b6-4f73-a869-64897464dcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:01.521246 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a93026-d8b6-4f73-a869-64897464dcb4 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.521680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a93026-d8b6-4f73-a869-64897464dcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.522124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a93026-d8b6-4f73-a869-64897464dcb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.540427 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a93026-d8b6-4f73-a869-64897464dcb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-34a93026-d8b6-4f73-a869-64897464dcb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:01.545693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:01.546910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:01.547948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:01.547948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:01.547948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:01.547948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:01.547948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:01.547948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:01.547948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:01.547948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:01.547948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:01.547948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:01.547948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:01.547948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:01.549799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 383/787] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:29:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:01.555166 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d48a1f-fdf7-4fc4-a435-cfd66e26a6c4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:01.614587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a93026-d8b6-4f73-a869-64897464dcb4 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:01.719038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d48a1f-fdf7-4fc4-a435-cfd66e26a6c4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:01.755142 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a93026-d8b6-4f73-a869-64897464dcb4 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:01.758877 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944633d7-8597-4bf1-b012-14efbe0ffe5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:01.759692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944633d7-8597-4bf1-b012-14efbe0ffe5c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.760123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944633d7-8597-4bf1-b012-14efbe0ffe5c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.760591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944633d7-8597-4bf1-b012-14efbe0ffe5c None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.761984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944633d7-8597-4bf1-b012-14efbe0ffe5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:01.762454 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944633d7-8597-4bf1-b012-14efbe0ffe5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:01.763993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944633d7-8597-4bf1-b012-14efbe0ffe5c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:01.764759 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:01.773425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 405/788] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:29:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:01.774096 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944633d7-8597-4bf1-b012-14efbe0ffe5c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:01.795511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5d3b9-55ec-4974-9b75-449aa3228ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:01.796303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5d3b9-55ec-4974-9b75-449aa3228ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.796623 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5d3b9-55ec-4974-9b75-449aa3228ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.796901 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5d3b9-55ec-4974-9b75-449aa3228ee0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.810432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5d3b9-55ec-4974-9b75-449aa3228ee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-4cd5d3b9-55ec-4974-9b75-449aa3228ee0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:01.815539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:01.816511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:01.817421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:01.817421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:01.817421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:01.817421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:01.817421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:01.817421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:01.817421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:01.817421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:01.817421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:01.817421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:01.817421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:29:01.817421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:01.817421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 384/789] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:29:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:01.818123 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5d3b9-55ec-4974-9b75-449aa3228ee0 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:01.839957 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:01.941339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944633d7-8597-4bf1-b012-14efbe0ffe5c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:01.952131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5d3b9-55ec-4974-9b75-449aa3228ee0 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:01.993078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b93cb3-0b99-4689-aece-a71233f1f11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:01.993296 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0b50c-5dfc-4260-ad99-78842ca24411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:01.993801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0b50c-5dfc-4260-ad99-78842ca24411 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.994083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0b50c-5dfc-4260-ad99-78842ca24411 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.994359 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0b50c-5dfc-4260-ad99-78842ca24411 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.994576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b93cb3-0b99-4689-aece-a71233f1f11b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.994959 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0b50c-5dfc-4260-ad99-78842ca24411 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:01.995173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b93cb3-0b99-4689-aece-a71233f1f11b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.995586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b93cb3-0b99-4689-aece-a71233f1f11b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.996998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0b50c-5dfc-4260-ad99-78842ca24411 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-43b0b50c-5dfc-4260-ad99-78842ca24411 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:02.008648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:02.009745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:02.010600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:02.010600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:02.010600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:02.010600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:02.010600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:02.010600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:02.010600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:02.010600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:02.010600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:02.010600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:02.010600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:02.010600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:02.010600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:29:02.010600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:02.010600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 406/790] 10.222.0.70 () {56 vars in 1129 bytes} [Wed Aug 24 16:29:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:02.011434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0b50c-5dfc-4260-ad99-78842ca24411 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:02.013081 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b93cb3-0b99-4689-aece-a71233f1f11b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:02.014798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b93cb3-0b99-4689-aece-a71233f1f11b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-86b93cb3-0b99-4689-aece-a71233f1f11b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:02.037923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:02.039012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:02.039922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:02.039922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:02.039922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:02.042157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 385/791] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:02.043833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b93cb3-0b99-4689-aece-a71233f1f11b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:02.066864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:02.169578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0b50c-5dfc-4260-ad99-78842ca24411 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:02.177441 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b93cb3-0b99-4689-aece-a71233f1f11b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:02.180920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:02.216509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be354f-400c-4cb5-8557-0dce6bdb4e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:02.217127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be354f-400c-4cb5-8557-0dce6bdb4e98 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:02.217413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be354f-400c-4cb5-8557-0dce6bdb4e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:02.217689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be354f-400c-4cb5-8557-0dce6bdb4e98 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:02.218362 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be354f-400c-4cb5-8557-0dce6bdb4e98 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:02.218667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be354f-400c-4cb5-8557-0dce6bdb4e98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:02.219219 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206be09-fe20-44ca-ab63-b2180928fbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:02.220018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206be09-fe20-44ca-ab63-b2180928fbef None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:02.220123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be354f-400c-4cb5-8557-0dce6bdb4e98 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:02.220626 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206be09-fe20-44ca-ab63-b2180928fbef None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:02.221254 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206be09-fe20-44ca-ab63-b2180928fbef None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:02.222087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206be09-fe20-44ca-ab63-b2180928fbef None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:02.222573 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206be09-fe20-44ca-ab63-b2180928fbef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:02.224119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206be09-fe20-44ca-ab63-b2180928fbef None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:02.229810 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 386/792] 10.222.0.70 () {56 vars in 1124 bytes} [Wed Aug 24 16:29:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:02.234866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 407/793] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:29:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:02.234866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be354f-400c-4cb5-8557-0dce6bdb4e98 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:02.236114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206be09-fe20-44ca-ab63-b2180928fbef None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:02.345994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be354f-400c-4cb5-8557-0dce6bdb4e98 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:02.352058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:02.380475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206be09-fe20-44ca-ab63-b2180928fbef None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:02.381638 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847b011-c941-46d2-81a2-ba2191438667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:02.382331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847b011-c941-46d2-81a2-ba2191438667 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:02.382575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847b011-c941-46d2-81a2-ba2191438667 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:02.382789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847b011-c941-46d2-81a2-ba2191438667 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:02.383429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847b011-c941-46d2-81a2-ba2191438667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:02.384467 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847b011-c941-46d2-81a2-ba2191438667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:02.385130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847b011-c941-46d2-81a2-ba2191438667 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:02.404309 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 408/794] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:02.404309 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847b011-c941-46d2-81a2-ba2191438667 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:02.434427 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8d26a-e221-4f68-b83f-1c26ed75358f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:02.435776 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8d26a-e221-4f68-b83f-1c26ed75358f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:02.436478 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8d26a-e221-4f68-b83f-1c26ed75358f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:02.436893 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8d26a-e221-4f68-b83f-1c26ed75358f None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:02.437854 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:02.439384 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8d26a-e221-4f68-b83f-1c26ed75358f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:02.440210 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8d26a-e221-4f68-b83f-1c26ed75358f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:02.441844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8d26a-e221-4f68-b83f-1c26ed75358f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:02.450297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 387/795] 10.222.0.70 () {56 vars in 1124 bytes} [Wed Aug 24 16:29:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:02.451392 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8d26a-e221-4f68-b83f-1c26ed75358f None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:02.523379 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847b011-c941-46d2-81a2-ba2191438667 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:02.583577 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f6887-8a55-4e86-9c68-f8864a04d1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:02.584345 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f6887-8a55-4e86-9c68-f8864a04d1ef None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:02.584733 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f6887-8a55-4e86-9c68-f8864a04d1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:02.585348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f6887-8a55-4e86-9c68-f8864a04d1ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:02.600668 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f6887-8a55-4e86-9c68-f8864a04d1ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:02.602272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f6887-8a55-4e86-9c68-f8864a04d1ef None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:02.616957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8d26a-e221-4f68-b83f-1c26ed75358f None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-3f8f6887-8a55-4e86-9c68-f8864a04d1ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:02.622445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:02.623439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:02.624312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:02.624312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:02.624312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:02.624312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 409/796] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:02.625420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f6887-8a55-4e86-9c68-f8864a04d1ef None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:02.658979 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3122582-4088-4bba-8f0a-e5fef2bb8261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:02.659912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3122582-4088-4bba-8f0a-e5fef2bb8261 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:02.660395 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3122582-4088-4bba-8f0a-e5fef2bb8261 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:02.660914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3122582-4088-4bba-8f0a-e5fef2bb8261 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:02.669882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3122582-4088-4bba-8f0a-e5fef2bb8261 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:02.676571 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3122582-4088-4bba-8f0a-e5fef2bb8261 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-f3122582-4088-4bba-8f0a-e5fef2bb8261 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:02.685157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:02.686177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:02.687103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:02.687103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:02.687103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:02.687103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:02.687103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:02.687103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:02.687103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:02.687103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:02.687103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:29:02.687103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:02.694461 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 388/797] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:29:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:02.695726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3122582-4088-4bba-8f0a-e5fef2bb8261 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:02.711239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:02.739978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:02.759277 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f6887-8a55-4e86-9c68-f8864a04d1ef None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:02.797803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16891018-e046-40f4-8572-10a2e5714640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:02.798479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16891018-e046-40f4-8572-10a2e5714640 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:02.798767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16891018-e046-40f4-8572-10a2e5714640 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:02.799037 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16891018-e046-40f4-8572-10a2e5714640 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:02.799700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16891018-e046-40f4-8572-10a2e5714640 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:02.801094 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16891018-e046-40f4-8572-10a2e5714640 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:02.833732 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3122582-4088-4bba-8f0a-e5fef2bb8261 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:02.870130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898528b-0b7b-4d5c-9df0-f3b43d5ee0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:02.871212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898528b-0b7b-4d5c-9df0-f3b43d5ee0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:02.871698 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898528b-0b7b-4d5c-9df0-f3b43d5ee0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:02.872227 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898528b-0b7b-4d5c-9df0-f3b43d5ee0f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:02.873090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898528b-0b7b-4d5c-9df0-f3b43d5ee0f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:02.873713 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898528b-0b7b-4d5c-9df0-f3b43d5ee0f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:02.875395 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898528b-0b7b-4d5c-9df0-f3b43d5ee0f6 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:02.889147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 389/798] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:02.894113 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898528b-0b7b-4d5c-9df0-f3b43d5ee0f6 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:02.933369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 410/799] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:29:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:02.934974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16891018-e046-40f4-8572-10a2e5714640 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:03.021988 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:03.073230 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898528b-0b7b-4d5c-9df0-f3b43d5ee0f6 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:03.136895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a461e-e30a-4e40-993e-9dc9625f7cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:03.137673 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a461e-e30a-4e40-993e-9dc9625f7cdc None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:03.138022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a461e-e30a-4e40-993e-9dc9625f7cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:03.138363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a461e-e30a-4e40-993e-9dc9625f7cdc None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:03.139177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a461e-e30a-4e40-993e-9dc9625f7cdc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:03.139553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a461e-e30a-4e40-993e-9dc9625f7cdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:03.143234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a461e-e30a-4e40-993e-9dc9625f7cdc None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:03.178807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 390/800] 10.222.0.70 () {56 vars in 1135 bytes} [Wed Aug 24 16:29:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:03.186507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a461e-e30a-4e40-993e-9dc9625f7cdc None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:03.320461 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a461e-e30a-4e40-993e-9dc9625f7cdc None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:03.347282 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:03.363013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16891018-e046-40f4-8572-10a2e5714640 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:03.372545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e75cd-d1c7-463f-9142-208c479456ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:03.373793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e75cd-d1c7-463f-9142-208c479456ae None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:03.374441 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e75cd-d1c7-463f-9142-208c479456ae None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:03.375082 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e75cd-d1c7-463f-9142-208c479456ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:03.399565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e75cd-d1c7-463f-9142-208c479456ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:03.402731 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e75cd-d1c7-463f-9142-208c479456ae None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:03.403425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8963c-767c-4fdf-bbc8-3a2fd3b5a00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:03.404295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8963c-767c-4fdf-bbc8-3a2fd3b5a00b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:03.404928 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8963c-767c-4fdf-bbc8-3a2fd3b5a00b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:03.405819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8963c-767c-4fdf-bbc8-3a2fd3b5a00b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:03.414476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a8963c-767c-4fdf-bbc8-3a2fd3b5a00b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:03.416262 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a8963c-767c-4fdf-bbc8-3a2fd3b5a00b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:03.419836 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-b5a8963c-767c-4fdf-bbc8-3a2fd3b5a00b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:03.430458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:03.431484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:03.432456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:03.432456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:03.432456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:03.432456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:03.432456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:03.432456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:03.432456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:03.432456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:03.432456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:29:03.432456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:03.442838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 411/801] 10.222.0.70 () {56 vars in 1117 bytes} [Wed Aug 24 16:29:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 501 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:03.442838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0a9bb-d48a-4470-9b7e-7b68f8e59b0e None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:03.442838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0a9bb-d48a-4470-9b7e-7b68f8e59b0e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:03.442838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0a9bb-d48a-4470-9b7e-7b68f8e59b0e None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:03.442838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 412/802] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-a67e75cd-d1c7-463f-9142-208c479456ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:03.443153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:03.444095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:03.445024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:03.445024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:03.445024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:03.445024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 391/803] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:03.448183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0a9bb-d48a-4470-9b7e-7b68f8e59b0e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:03.449703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426b391-4e2c-4a26-94c8-85acd164fe87 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:03.451158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426b391-4e2c-4a26-94c8-85acd164fe87 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:03.451645 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426b391-4e2c-4a26-94c8-85acd164fe87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:03.488834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-c426b391-4e2c-4a26-94c8-85acd164fe87 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:03.488834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c426b391-4e2c-4a26-94c8-85acd164fe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:03.579496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0a9bb-d48a-4470-9b7e-7b68f8e59b0e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:03.592452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 392/804] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:03.592452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426b391-4e2c-4a26-94c8-85acd164fe87 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:03.622807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a0ab3-4c55-42b2-b5fb-4d7b0cd9205e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:03.623744 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a0ab3-4c55-42b2-b5fb-4d7b0cd9205e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:03.624174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a0ab3-4c55-42b2-b5fb-4d7b0cd9205e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:03.624636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a0ab3-4c55-42b2-b5fb-4d7b0cd9205e None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:03.625526 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a0ab3-4c55-42b2-b5fb-4d7b0cd9205e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:03.626051 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a0ab3-4c55-42b2-b5fb-4d7b0cd9205e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:03.627754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a0ab3-4c55-42b2-b5fb-4d7b0cd9205e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:03.653019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 413/805] 10.222.0.70 () {56 vars in 1149 bytes} [Wed Aug 24 16:29:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:03.654892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8ca8e-4651-4636-9e1f-84d06b36c0f6 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:03.655349 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8ca8e-4651-4636-9e1f-84d06b36c0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:03.655900 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8ca8e-4651-4636-9e1f-84d06b36c0f6 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:03.659495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 414/806] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:03.660803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8ca8e-4651-4636-9e1f-84d06b36c0f6 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:03.699403 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426b391-4e2c-4a26-94c8-85acd164fe87 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:03.707694 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:03.726420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:03.738061 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d15919-55cc-4f93-a95e-734b0fbba711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:03.738747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d15919-55cc-4f93-a95e-734b0fbba711 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:03.739027 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d15919-55cc-4f93-a95e-734b0fbba711 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:03.739300 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d15919-55cc-4f93-a95e-734b0fbba711 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:03.740009 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d15919-55cc-4f93-a95e-734b0fbba711 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:03.740782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d15919-55cc-4f93-a95e-734b0fbba711 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:03.742267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d15919-55cc-4f93-a95e-734b0fbba711 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:03.755891 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 393/807] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:03.757305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad3ab4-4759-4f18-89a0-3f850cde1178 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:03.757655 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad3ab4-4759-4f18-89a0-3f850cde1178 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:03.757940 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad3ab4-4759-4f18-89a0-3f850cde1178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:03.779298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8ca8e-4651-4636-9e1f-84d06b36c0f6 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:03.779632 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-b0ad3ab4-4759-4f18-89a0-3f850cde1178 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:03.780969 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b0ad3ab4-4759-4f18-89a0-3f850cde1178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:03.838785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfb766-c33d-4d61-8f90-8601d7e3b8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:03.840886 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfb766-c33d-4d61-8f90-8601d7e3b8dd None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:03.841944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfb766-c33d-4d61-8f90-8601d7e3b8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:03.842524 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfb766-c33d-4d61-8f90-8601d7e3b8dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:03.858511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 394/808] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:03.859571 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad3ab4-4759-4f18-89a0-3f850cde1178 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:03.864016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfb766-c33d-4d61-8f90-8601d7e3b8dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:03.865692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfb766-c33d-4d61-8f90-8601d7e3b8dd None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-79cfb766-c33d-4d61-8f90-8601d7e3b8dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:03.886962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:03.887975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:03.888911 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:03.888911 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:03.888911 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:03.889178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 415/809] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:03.895035 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfb766-c33d-4d61-8f90-8601d7e3b8dd None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:03.964783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad3ab4-4759-4f18-89a0-3f850cde1178 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:03.972738 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:04.002546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a36477-70cd-4bb3-ac44-07fa823ce824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.002769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfb766-c33d-4d61-8f90-8601d7e3b8dd None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.003565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a36477-70cd-4bb3-ac44-07fa823ce824 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.003946 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a36477-70cd-4bb3-ac44-07fa823ce824 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.004319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a36477-70cd-4bb3-ac44-07fa823ce824 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.012760 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a36477-70cd-4bb3-ac44-07fa823ce824 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:04.014445 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a36477-70cd-4bb3-ac44-07fa823ce824 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:04.015268 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:04.027590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-50a36477-70cd-4bb3-ac44-07fa823ce824 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:04.027590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:04.027590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:04.027590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:04.027590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:04.027590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:04.027590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:04.027590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:04.027590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:04.027590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:04.027590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:04.027590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:04.027590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:04.027590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:04.027590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:04.028774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:04.029941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:04.029941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:04.029941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:04.029941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:04.029941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:04.029941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:04.029941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:04.029941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:04.031415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 395/810] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:04.032076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a36477-70cd-4bb3-ac44-07fa823ce824 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.046318 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d4b19-7ac3-481f-993f-c8871292d6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.047311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d4b19-7ac3-481f-993f-c8871292d6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.047810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d4b19-7ac3-481f-993f-c8871292d6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.048276 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d4b19-7ac3-481f-993f-c8871292d6f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.049244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d4b19-7ac3-481f-993f-c8871292d6f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:04.049796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d4b19-7ac3-481f-993f-c8871292d6f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:04.051341 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d4b19-7ac3-481f-993f-c8871292d6f9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:04.070464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 416/811] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:04.071730 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d4b19-7ac3-481f-993f-c8871292d6f9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.146134 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a36477-70cd-4bb3-ac44-07fa823ce824 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.177916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d4b19-7ac3-481f-993f-c8871292d6f9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.182138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859711c9-5a19-49d4-a8d3-5b07869a3750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.182850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859711c9-5a19-49d4-a8d3-5b07869a3750 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.183746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859711c9-5a19-49d4-a8d3-5b07869a3750 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.184107 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859711c9-5a19-49d4-a8d3-5b07869a3750 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.196923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859711c9-5a19-49d4-a8d3-5b07869a3750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-859711c9-5a19-49d4-a8d3-5b07869a3750 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:04.203080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:04.204044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:04.204935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:04.204935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:04.204935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:04.204935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:04.204935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:04.204935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:04.204935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:04.204935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:04.204935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:04.204935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:04.204935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:04.204935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:04.211493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 396/812] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:29:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:04.211493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859711c9-5a19-49d4-a8d3-5b07869a3750 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.216087 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69f033-d93a-46d1-a79b-b07a334863bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.216666 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69f033-d93a-46d1-a79b-b07a334863bb None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.216947 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69f033-d93a-46d1-a79b-b07a334863bb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.217217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69f033-d93a-46d1-a79b-b07a334863bb None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.217915 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69f033-d93a-46d1-a79b-b07a334863bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:04.218221 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69f033-d93a-46d1-a79b-b07a334863bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:04.219713 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69f033-d93a-46d1-a79b-b07a334863bb None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:04.228107 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:04.236456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 417/813] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:04.236456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69f033-d93a-46d1-a79b-b07a334863bb None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.267468 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:04.327897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859711c9-5a19-49d4-a8d3-5b07869a3750 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.341607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69f033-d93a-46d1-a79b-b07a334863bb None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.373995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae3309-e8fd-40c8-bef5-de8998f2d9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.374843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae3309-e8fd-40c8-bef5-de8998f2d9bc None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.375381 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae3309-e8fd-40c8-bef5-de8998f2d9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.375499 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae3309-e8fd-40c8-bef5-de8998f2d9bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.381963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a46d0-d3e7-432b-988b-a2398604855d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.382591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a46d0-d3e7-432b-988b-a2398604855d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.382869 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a46d0-d3e7-432b-988b-a2398604855d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.383141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a46d0-d3e7-432b-988b-a2398604855d None admin] PATH_INFO: `/v3/services/network` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.383761 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a46d0-d3e7-432b-988b-a2398604855d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:04.384641 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae3309-e8fd-40c8-bef5-de8998f2d9bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:04.385170 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a46d0-d3e7-432b-988b-a2398604855d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:04.386066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae3309-e8fd-40c8-bef5-de8998f2d9bc None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-702a46d0-d3e7-432b-988b-a2398604855d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:04.394330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:04.395322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:04.401344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:04.401344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:04.401344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:04.401344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:04.401344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:04.401344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:04.401344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:04.401344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:04.401344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:04.401344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:04.401344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:04.401344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:04.401344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:29:04.401344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:04.401344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 418/814] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:04 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:04.402191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a46d0-d3e7-432b-988b-a2398604855d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-28ae3309-e8fd-40c8-bef5-de8998f2d9bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:04.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:04.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:04.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:04.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:04.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:04.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:04.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:04.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:04.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:04.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:04.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:04.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:04.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:04.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:04.403540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:04.405187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:04.405187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:04.405187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:04.405187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:04.405187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:04.405187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:04.405187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:04.405187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:04.405187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 397/815] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:04.405187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae3309-e8fd-40c8-bef5-de8998f2d9bc None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.488782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:04.503010 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:04.514158 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a46d0-d3e7-432b-988b-a2398604855d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.531686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae3309-e8fd-40c8-bef5-de8998f2d9bc None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.562619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45926374-d14b-4567-a6c5-00489ec02da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.563243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45926374-d14b-4567-a6c5-00489ec02da5 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.563527 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45926374-d14b-4567-a6c5-00489ec02da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.563801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45926374-d14b-4567-a6c5-00489ec02da5 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.564768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45926374-d14b-4567-a6c5-00489ec02da5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:04.565016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45926374-d14b-4567-a6c5-00489ec02da5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:04.566351 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45926374-d14b-4567-a6c5-00489ec02da5 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:04.575195 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785707c1-1f57-4494-9053-0e7ccb84ecd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.580378 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 419/816] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:29:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:04.580605 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785707c1-1f57-4494-9053-0e7ccb84ecd4 None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.580605 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785707c1-1f57-4494-9053-0e7ccb84ecd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.580605 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785707c1-1f57-4494-9053-0e7ccb84ecd4 None admin] PATH_INFO: `/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/b8ec0b1e5f4d434bb725297f3f867ba7/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.580865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45926374-d14b-4567-a6c5-00489ec02da5 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.582444 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785707c1-1f57-4494-9053-0e7ccb84ecd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc920dce30734ac7a69dc039d8c74ce1, user_id=b8ec0b1e5f4d434bb725297f3f867ba7, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:04.583996 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785707c1-1f57-4494-9053-0e7ccb84ecd4 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:04.640419 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 398/817] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:29:04 2022] PUT /identity/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/b8ec0b1e5f4d434bb725297f3f867ba7/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:29:04.641577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785707c1-1f57-4494-9053-0e7ccb84ecd4 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.725307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45926374-d14b-4567-a6c5-00489ec02da5 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.781958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d0328-e854-4035-9af6-0118b1c334b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.782694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d0328-e854-4035-9af6-0118b1c334b3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.783037 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d0328-e854-4035-9af6-0118b1c334b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.783423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d0328-e854-4035-9af6-0118b1c334b3 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.784140 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d0328-e854-4035-9af6-0118b1c334b3 None admin] RBAC: Adding query filter params (name=network) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:04.784478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d0328-e854-4035-9af6-0118b1c334b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:04.786038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d0328-e854-4035-9af6-0118b1c334b3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:04.791671 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 420/818] 10.222.0.70 () {56 vars in 1112 bytes} [Wed Aug 24 16:29:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:04.793143 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d0328-e854-4035-9af6-0118b1c334b3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.817184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:04.822697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785707c1-1f57-4494-9053-0e7ccb84ecd4 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.860194 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:04.867145 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2c31f-f478-4a62-bdf2-f8c21c7ff086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.867860 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2c31f-f478-4a62-bdf2-f8c21c7ff086 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.868198 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2c31f-f478-4a62-bdf2-f8c21c7ff086 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.869005 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2c31f-f478-4a62-bdf2-f8c21c7ff086 None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.869909 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2c31f-f478-4a62-bdf2-f8c21c7ff086 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:04.869909 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2c31f-f478-4a62-bdf2-f8c21c7ff086 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:04.871458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2c31f-f478-4a62-bdf2-f8c21c7ff086 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:04.887308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 399/819] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:04.889093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2c31f-f478-4a62-bdf2-f8c21c7ff086 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.934857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d0328-e854-4035-9af6-0118b1c334b3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.976662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d76f5-589c-4d3c-a382-8fe732f4b482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.977623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d76f5-589c-4d3c-a382-8fe732f4b482 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.978020 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d76f5-589c-4d3c-a382-8fe732f4b482 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.978409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d76f5-589c-4d3c-a382-8fe732f4b482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.991789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d76f5-589c-4d3c-a382-8fe732f4b482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:04.993551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d76f5-589c-4d3c-a382-8fe732f4b482 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-965d76f5-589c-4d3c-a382-8fe732f4b482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:05.009935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:05.011963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:05.013368 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:05.013368 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:05.013368 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:05.013368 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 421/820] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:05.013593 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d76f5-589c-4d3c-a382-8fe732f4b482 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:05.027423 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2c31f-f478-4a62-bdf2-f8c21c7ff086 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:05.081305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876da9d-5883-4eeb-9d4b-5f5c28e96bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:05.082513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876da9d-5883-4eeb-9d4b-5f5c28e96bec None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:05.083109 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876da9d-5883-4eeb-9d4b-5f5c28e96bec None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:05.083810 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876da9d-5883-4eeb-9d4b-5f5c28e96bec None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:05.085092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876da9d-5883-4eeb-9d4b-5f5c28e96bec None admin] RBAC: Adding query filter params (type=network) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:05.085708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876da9d-5883-4eeb-9d4b-5f5c28e96bec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:05.088559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876da9d-5883-4eeb-9d4b-5f5c28e96bec None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:05.104756 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 400/821] 10.222.0.70 () {56 vars in 1112 bytes} [Wed Aug 24 16:29:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:05.106370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876da9d-5883-4eeb-9d4b-5f5c28e96bec None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:05.112944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:05.145709 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d76f5-589c-4d3c-a382-8fe732f4b482 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:05.181858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e8598-b99a-4a87-9116-ac899c1b2de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:05.182533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e8598-b99a-4a87-9116-ac899c1b2de2 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:05.182818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e8598-b99a-4a87-9116-ac899c1b2de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:05.183135 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e8598-b99a-4a87-9116-ac899c1b2de2 None admin] PATH_INFO: `/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/7f3899f2eb554f2fb5a7c43f660b51d5/roles/bb34a44869834493b10a99cc751ab6c2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:05.200215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e8598-b99a-4a87-9116-ac899c1b2de2 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc920dce30734ac7a69dc039d8c74ce1, user_id=7f3899f2eb554f2fb5a7c43f660b51d5, role_id=bb34a44869834493b10a99cc751ab6c2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:05.201692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e8598-b99a-4a87-9116-ac899c1b2de2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:05.213680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:05.280526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876da9d-5883-4eeb-9d4b-5f5c28e96bec None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:05.288646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 422/822] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:29:05 2022] PUT /identity/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/7f3899f2eb554f2fb5a7c43f660b51d5/roles/bb34a44869834493b10a99cc751ab6c2 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:29:05.288646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e8598-b99a-4a87-9116-ac899c1b2de2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:05.347863 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7577e9-0cdb-4e6d-b4d4-fc2497446e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:05.348639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7577e9-0cdb-4e6d-b4d4-fc2497446e3c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:05.349111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7577e9-0cdb-4e6d-b4d4-fc2497446e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:05.349568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7577e9-0cdb-4e6d-b4d4-fc2497446e3c None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:05.350679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7577e9-0cdb-4e6d-b4d4-fc2497446e3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:05.351206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7577e9-0cdb-4e6d-b4d4-fc2497446e3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:05.353755 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7577e9-0cdb-4e6d-b4d4-fc2497446e3c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:05.377056 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 401/823] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:05.379251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9ba73-b1c5-471a-8a08-3f77e4ca78aa None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:05.379725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9ba73-b1c5-471a-8a08-3f77e4ca78aa None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:05.380185 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9ba73-b1c5-471a-8a08-3f77e4ca78aa None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:05.388628 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 402/824] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:05 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:05.390109 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9ba73-b1c5-471a-8a08-3f77e4ca78aa None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:05.450902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e8598-b99a-4a87-9116-ac899c1b2de2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:05.481454 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:05.498781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b5239-fb7f-45d4-a112-98f69d3ba49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:05.499399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b5239-fb7f-45d4-a112-98f69d3ba49f None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:05.499675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b5239-fb7f-45d4-a112-98f69d3ba49f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:05.500058 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b5239-fb7f-45d4-a112-98f69d3ba49f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:05.510230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b5239-fb7f-45d4-a112-98f69d3ba49f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:05.510230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b5239-fb7f-45d4-a112-98f69d3ba49f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:05.528941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9ba73-b1c5-471a-8a08-3f77e4ca78aa None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:05.549563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:05.570784 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544dcba-d146-4035-b8b1-218773514863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:05.571560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544dcba-d146-4035-b8b1-218773514863 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:05.571749 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544dcba-d146-4035-b8b1-218773514863 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:05.572017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544dcba-d146-4035-b8b1-218773514863 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:05.580719 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544dcba-d146-4035-b8b1-218773514863 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:05.582204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544dcba-d146-4035-b8b1-218773514863 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-3544dcba-d146-4035-b8b1-218773514863 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:05.593592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:05.594544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:05.595493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:05.595493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:05.595493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:05.595493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:05.595493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:05.595493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:05.595493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:05.595493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:05.595493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:29:05.595493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:05.595493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 403/825] 10.222.0.70 () {56 vars in 1093 bytes} [Wed Aug 24 16:29:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:05.602242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933090ff-6ef7-473a-8be4-a28c39a5c7ed None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:05.602242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933090ff-6ef7-473a-8be4-a28c39a5c7ed None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:05.602242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933090ff-6ef7-473a-8be4-a28c39a5c7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:05.606932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 423/826] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:29:05 2022] POST /identity/v3/endpoints => generated 337 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:05.612468 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b5239-fb7f-45d4-a112-98f69d3ba49f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:05.636812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-933090ff-6ef7-473a-8be4-a28c39a5c7ed None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:05.640056 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-933090ff-6ef7-473a-8be4-a28c39a5c7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:05.728758 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 404/827] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:05.810585 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b5239-fb7f-45d4-a112-98f69d3ba49f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:05.848503 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ded408-c06e-476b-9a26-f5d8e16b9b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:05.849188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ded408-c06e-476b-9a26-f5d8e16b9b9d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:05.849479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ded408-c06e-476b-9a26-f5d8e16b9b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:05.849752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ded408-c06e-476b-9a26-f5d8e16b9b9d None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:05.850493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ded408-c06e-476b-9a26-f5d8e16b9b9d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:05.850852 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ded408-c06e-476b-9a26-f5d8e16b9b9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:05.852354 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ded408-c06e-476b-9a26-f5d8e16b9b9d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:05.873742 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 424/828] 10.222.0.70 () {56 vars in 1133 bytes} [Wed Aug 24 16:29:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:05.882964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933090ff-6ef7-473a-8be4-a28c39a5c7ed None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:06.062686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933090ff-6ef7-473a-8be4-a28c39a5c7ed None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:06.065784 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:06.084712 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ded408-c06e-476b-9a26-f5d8e16b9b9d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:06.133327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:06.134334 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c6d5b-f590-4470-a0b8-ff35188eaa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:06.135360 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c6d5b-f590-4470-a0b8-ff35188eaa27 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:06.135858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c6d5b-f590-4470-a0b8-ff35188eaa27 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:06.136365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c6d5b-f590-4470-a0b8-ff35188eaa27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:06.251163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ded408-c06e-476b-9a26-f5d8e16b9b9d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:06.257593 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c6d5b-f590-4470-a0b8-ff35188eaa27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:06.259065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c6d5b-f590-4470-a0b8-ff35188eaa27 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:06.314476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fe83b-acd8-4796-9c7c-53358912080e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:06.315667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fe83b-acd8-4796-9c7c-53358912080e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:06.316270 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fe83b-acd8-4796-9c7c-53358912080e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:06.316894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fe83b-acd8-4796-9c7c-53358912080e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:06.318091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169fe83b-acd8-4796-9c7c-53358912080e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:06.321141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169fe83b-acd8-4796-9c7c-53358912080e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-169fe83b-acd8-4796-9c7c-53358912080e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:06.345053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:06.346330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:06.347338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:06.347338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:06.347338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:06.347338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:06.347338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:06.347338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:06.347338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:06.347338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:06.347338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:06.347338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:06.347338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:06.347338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:06.347338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:29:06.347338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:06.355620 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 425/829] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:29:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:06.361537 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fe83b-acd8-4796-9c7c-53358912080e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:06.399545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 405/830] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:05 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:29:06.402861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c6d5b-f590-4470-a0b8-ff35188eaa27 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:06.405387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:06.521585 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c6d5b-f590-4470-a0b8-ff35188eaa27 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:06.556911 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:06.562369 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad05bd1-6b11-48f1-adaa-b1d1021b8ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:06.563071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad05bd1-6b11-48f1-adaa-b1d1021b8ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:06.563357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad05bd1-6b11-48f1-adaa-b1d1021b8ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:06.563629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad05bd1-6b11-48f1-adaa-b1d1021b8ca9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:06.568135 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fe83b-acd8-4796-9c7c-53358912080e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:06.576493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad05bd1-6b11-48f1-adaa-b1d1021b8ca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:06.577990 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad05bd1-6b11-48f1-adaa-b1d1021b8ca9 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-cad05bd1-6b11-48f1-adaa-b1d1021b8ca9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:06.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:06.597065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:06.598769 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:06.598769 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:06.598769 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:06.600887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 406/831] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:06.602297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad05bd1-6b11-48f1-adaa-b1d1021b8ca9 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:06.647090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab36062-c4b6-4abf-be70-5b6ff695a7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:06.647747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab36062-c4b6-4abf-be70-5b6ff695a7ea None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:06.648029 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab36062-c4b6-4abf-be70-5b6ff695a7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:06.648305 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab36062-c4b6-4abf-be70-5b6ff695a7ea None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:06.648999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab36062-c4b6-4abf-be70-5b6ff695a7ea None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:06.649294 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab36062-c4b6-4abf-be70-5b6ff695a7ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:06.650822 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab36062-c4b6-4abf-be70-5b6ff695a7ea None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:06.660511 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 426/832] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:29:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:06.665866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab36062-c4b6-4abf-be70-5b6ff695a7ea None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:06.693608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:06.745247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad05bd1-6b11-48f1-adaa-b1d1021b8ca9 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:06.786906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776be5e7-9b8f-474a-925f-bd385ed03abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:06.787554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776be5e7-9b8f-474a-925f-bd385ed03abd None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:06.787836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776be5e7-9b8f-474a-925f-bd385ed03abd None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:06.788128 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776be5e7-9b8f-474a-925f-bd385ed03abd None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:06.788983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776be5e7-9b8f-474a-925f-bd385ed03abd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:06.789293 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776be5e7-9b8f-474a-925f-bd385ed03abd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:06.791078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776be5e7-9b8f-474a-925f-bd385ed03abd None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:06.797612 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab36062-c4b6-4abf-be70-5b6ff695a7ea None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:06.808529 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 407/833] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:06.810388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776be5e7-9b8f-474a-925f-bd385ed03abd None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:06.837462 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc00648-71dd-430a-8809-4a190e9f2016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:06.838543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc00648-71dd-430a-8809-4a190e9f2016 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:06.839031 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc00648-71dd-430a-8809-4a190e9f2016 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:06.840085 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc00648-71dd-430a-8809-4a190e9f2016 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:06.841017 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc00648-71dd-430a-8809-4a190e9f2016 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:06.841628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc00648-71dd-430a-8809-4a190e9f2016 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:06.843343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc00648-71dd-430a-8809-4a190e9f2016 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:06.849315 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 427/834] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:29:06 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:06.894368 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043debd-9d30-44bc-adb0-829c7b899bf1 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:06.894736 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043debd-9d30-44bc-adb0-829c7b899bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:06.895135 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043debd-9d30-44bc-adb0-829c7b899bf1 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:06.900457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 428/835] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:29:06.903615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266be94-8304-4a98-a559-67bc177edc7d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:06.904193 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266be94-8304-4a98-a559-67bc177edc7d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:06.904689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266be94-8304-4a98-a559-67bc177edc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:06.925507 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e266be94-8304-4a98-a559-67bc177edc7d None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:06.927126 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e266be94-8304-4a98-a559-67bc177edc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:06.959470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:06.963795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776be5e7-9b8f-474a-925f-bd385ed03abd None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:06.984599 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:07.015089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c0514-0b98-45de-b5f1-71d8a72e2145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:07.016831 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c0514-0b98-45de-b5f1-71d8a72e2145 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:07.017406 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c0514-0b98-45de-b5f1-71d8a72e2145 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:07.017847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c0514-0b98-45de-b5f1-71d8a72e2145 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:07.027683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c0514-0b98-45de-b5f1-71d8a72e2145 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:07.029485 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 429/836] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:29:07.029779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c0514-0b98-45de-b5f1-71d8a72e2145 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:07.046136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-c09c0514-0b98-45de-b5f1-71d8a72e2145 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:07.046136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:07.046136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:07.046136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:07.046136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:07.046136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:07.046136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:07.046136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:07.046136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:07.046136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:07.046136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:07.046136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:07.046136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:07.046136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:07.046136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:07.047092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:07.047991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:07.047991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:07.047991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:07.047991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:07.047991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:07.047991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:07.047991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:07.047991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:07.048718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 408/837] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:07.056066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266be94-8304-4a98-a559-67bc177edc7d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:07.189616 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266be94-8304-4a98-a559-67bc177edc7d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:07.230978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef996d-f919-42ca-bf43-828b143fcac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:07.231608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef996d-f919-42ca-bf43-828b143fcac7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:07.231888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef996d-f919-42ca-bf43-828b143fcac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:07.232160 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef996d-f919-42ca-bf43-828b143fcac7 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:07.232908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef996d-f919-42ca-bf43-828b143fcac7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:07.233222 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef996d-f919-42ca-bf43-828b143fcac7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:07.234675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef996d-f919-42ca-bf43-828b143fcac7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:07.248906 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 430/838] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:07.255085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c0514-0b98-45de-b5f1-71d8a72e2145 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:07.329839 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef996d-f919-42ca-bf43-828b143fcac7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:07.351149 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:07.427079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c0514-0b98-45de-b5f1-71d8a72e2145 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:07.459057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef996d-f919-42ca-bf43-828b143fcac7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:07.468879 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070576a-dd36-47b8-b6c1-3f6ab776d8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:07.469631 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070576a-dd36-47b8-b6c1-3f6ab776d8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:07.470144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070576a-dd36-47b8-b6c1-3f6ab776d8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:07.470664 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070576a-dd36-47b8-b6c1-3f6ab776d8b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:07.472131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e070576a-dd36-47b8-b6c1-3f6ab776d8b4 None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=e333fca7790244f6a4ebceabb55faba4) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:07.472878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e070576a-dd36-47b8-b6c1-3f6ab776d8b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:07.474851 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e070576a-dd36-47b8-b6c1-3f6ab776d8b4 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:07.482829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 409/839] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:07 2022] GET /identity/v3/role_assignments?user.id=e333fca7790244f6a4ebceabb55faba4&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=e7f48343fc1542979ab4459865109e30 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:07.506791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf99217-89ee-465d-8155-1fc6ab9e24ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:07.507528 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf99217-89ee-465d-8155-1fc6ab9e24ff None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:07.507846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf99217-89ee-465d-8155-1fc6ab9e24ff None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:07.508116 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf99217-89ee-465d-8155-1fc6ab9e24ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:07.528700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf99217-89ee-465d-8155-1fc6ab9e24ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-dcf99217-89ee-465d-8155-1fc6ab9e24ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:07.539078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:07.541002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:07.545055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:07.545055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:07.545055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:07.545055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:07.545055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:07.545055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:07.545055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:07.545055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:07.545055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:07.545055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:07.545055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:07.545055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:07.545055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 431/840] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:29:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:07.551137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e97ad-c5a5-4322-bcbd-52a1b00e83cf None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:07.552183 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e97ad-c5a5-4322-bcbd-52a1b00e83cf None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:07.552183 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e97ad-c5a5-4322-bcbd-52a1b00e83cf None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:07.552615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf99217-89ee-465d-8155-1fc6ab9e24ff None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:07.554850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 410/841] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:07.567007 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc0736-8a0e-4afa-abef-0a01c26f2f24 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:07.567366 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc0736-8a0e-4afa-abef-0a01c26f2f24 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:07.567581 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc0736-8a0e-4afa-abef-0a01c26f2f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:07.591489 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-c1cc0736-8a0e-4afa-abef-0a01c26f2f24 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:07.593066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c1cc0736-8a0e-4afa-abef-0a01c26f2f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:07.633959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:07.652635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:07.680017 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf99217-89ee-465d-8155-1fc6ab9e24ff None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:07.688576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 411/842] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:29:07.725248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540f742-000f-48c5-a44f-dbfded4008eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:07.725689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540f742-000f-48c5-a44f-dbfded4008eb None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:07.725689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540f742-000f-48c5-a44f-dbfded4008eb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:07.726167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540f742-000f-48c5-a44f-dbfded4008eb None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:07.728871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540f742-000f-48c5-a44f-dbfded4008eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:07.728871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540f742-000f-48c5-a44f-dbfded4008eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:07.728871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540f742-000f-48c5-a44f-dbfded4008eb None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:07.738841 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 432/843] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:29:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:07.744755 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc0736-8a0e-4afa-abef-0a01c26f2f24 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:07.862687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc0736-8a0e-4afa-abef-0a01c26f2f24 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:07.902977 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e8044-2313-4c59-a501-a2fc61340392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:07.903849 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e8044-2313-4c59-a501-a2fc61340392 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:07.904131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e8044-2313-4c59-a501-a2fc61340392 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:07.904480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e8044-2313-4c59-a501-a2fc61340392 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:07.916378 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e8044-2313-4c59-a501-a2fc61340392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:07.918563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e8044-2313-4c59-a501-a2fc61340392 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-841e8044-2313-4c59-a501-a2fc61340392 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:07.935696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:07.938099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:07.940143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:07.940143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:07.940143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:07.940143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 412/844] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:07.943165 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540f742-000f-48c5-a44f-dbfded4008eb None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:07.972719 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e8044-2313-4c59-a501-a2fc61340392 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:08.005791 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:08.067759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540f742-000f-48c5-a44f-dbfded4008eb None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:08.109847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbce19-65c8-4012-9ffc-f39df67c486a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:08.110662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbce19-65c8-4012-9ffc-f39df67c486a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:08.110942 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbce19-65c8-4012-9ffc-f39df67c486a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:08.113840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbce19-65c8-4012-9ffc-f39df67c486a None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:08.115193 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbce19-65c8-4012-9ffc-f39df67c486a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:08.115757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbce19-65c8-4012-9ffc-f39df67c486a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:08.118572 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbce19-65c8-4012-9ffc-f39df67c486a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:08.132722 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:08.139720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 433/845] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:08.141452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e8044-2313-4c59-a501-a2fc61340392 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:08.145592 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbce19-65c8-4012-9ffc-f39df67c486a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:08.196090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d249cb2-5eaf-404b-b2cc-7ad5802f3230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:08.197228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d249cb2-5eaf-404b-b2cc-7ad5802f3230 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:08.197672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d249cb2-5eaf-404b-b2cc-7ad5802f3230 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:08.198107 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d249cb2-5eaf-404b-b2cc-7ad5802f3230 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:08.210367 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d249cb2-5eaf-404b-b2cc-7ad5802f3230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-6d249cb2-5eaf-404b-b2cc-7ad5802f3230 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:08.216865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:08.217900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:08.218751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:08.218751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:08.218751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:08.218751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:08.218751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:08.218751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:08.218751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:08.218751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:08.218751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:08.218751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:08.218751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:29:08.218751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:08.223007 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 413/846] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:29:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:08.224065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d249cb2-5eaf-404b-b2cc-7ad5802f3230 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:08.305876 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbce19-65c8-4012-9ffc-f39df67c486a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:08.346038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:08.350298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac3b85-f148-4fa4-83d0-0871cfef1de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:08.351121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac3b85-f148-4fa4-83d0-0871cfef1de9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:08.355529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac3b85-f148-4fa4-83d0-0871cfef1de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:08.355529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac3b85-f148-4fa4-83d0-0871cfef1de9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:08.362079 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac3b85-f148-4fa4-83d0-0871cfef1de9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:08.363762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac3b85-f148-4fa4-83d0-0871cfef1de9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:08.371480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d249cb2-5eaf-404b-b2cc-7ad5802f3230 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-5bac3b85-f148-4fa4-83d0-0871cfef1de9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:08.379520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:08.381665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:08.384998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:08.384998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:08.384998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:08.384998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:08.384998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:08.384998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:08.384998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:08.384998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:08.384998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:29:08.384998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:08.384998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 434/847] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:29:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:08.388665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac3b85-f148-4fa4-83d0-0871cfef1de9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:08.441524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648a930-ac37-4184-ae6d-66abe07a7a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:08.443379 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648a930-ac37-4184-ae6d-66abe07a7a15 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:08.443889 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648a930-ac37-4184-ae6d-66abe07a7a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:08.444388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648a930-ac37-4184-ae6d-66abe07a7a15 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:08.445786 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648a930-ac37-4184-ae6d-66abe07a7a15 None admin] RBAC: Adding query filter params (name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:08.446416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648a930-ac37-4184-ae6d-66abe07a7a15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:08.449465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648a930-ac37-4184-ae6d-66abe07a7a15 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:08.457760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:08.467422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 414/848] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:29:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:08.474680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648a930-ac37-4184-ae6d-66abe07a7a15 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:08.519413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac3b85-f148-4fa4-83d0-0871cfef1de9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:08.566999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0af82a-b7d7-40cb-9fb0-541518c11292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:08.567550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0af82a-b7d7-40cb-9fb0-541518c11292 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:08.567829 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0af82a-b7d7-40cb-9fb0-541518c11292 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:08.568099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0af82a-b7d7-40cb-9fb0-541518c11292 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:08.568879 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0af82a-b7d7-40cb-9fb0-541518c11292 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:08.569183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0af82a-b7d7-40cb-9fb0-541518c11292 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:08.570626 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0af82a-b7d7-40cb-9fb0-541518c11292 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:08.595418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 435/849] 10.222.0.70 () {56 vars in 1135 bytes} [Wed Aug 24 16:29:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:08.596877 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0af82a-b7d7-40cb-9fb0-541518c11292 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:08.629134 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648a930-ac37-4184-ae6d-66abe07a7a15 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:08.668511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c16fd8-4684-443d-8888-da6fa462e29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:08.671112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c16fd8-4684-443d-8888-da6fa462e29f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:08.671112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c16fd8-4684-443d-8888-da6fa462e29f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:08.671112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c16fd8-4684-443d-8888-da6fa462e29f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:08.682511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c16fd8-4684-443d-8888-da6fa462e29f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:08.684483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c16fd8-4684-443d-8888-da6fa462e29f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:08.686869 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-44c16fd8-4684-443d-8888-da6fa462e29f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:08.702238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:08.703234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:08.704144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:08.704144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:08.704144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:08.704814 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0af82a-b7d7-40cb-9fb0-541518c11292 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:08.707003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 415/850] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:08.712854 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c16fd8-4684-443d-8888-da6fa462e29f None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:08.726989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:08.751313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9750319-8cef-440e-9efd-bd483beec068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:08.752490 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9750319-8cef-440e-9efd-bd483beec068 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:08.752983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9750319-8cef-440e-9efd-bd483beec068 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:08.753515 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9750319-8cef-440e-9efd-bd483beec068 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:08.860665 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c16fd8-4684-443d-8888-da6fa462e29f None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:08.906554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9750319-8cef-440e-9efd-bd483beec068 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:08.908002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b4ff5-8486-4ad7-9c7c-70ca913d2411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:08.908921 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b4ff5-8486-4ad7-9c7c-70ca913d2411 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:08.909447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9750319-8cef-440e-9efd-bd483beec068 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:08.909761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b4ff5-8486-4ad7-9c7c-70ca913d2411 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:08.910246 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b4ff5-8486-4ad7-9c7c-70ca913d2411 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:08.911124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b4ff5-8486-4ad7-9c7c-70ca913d2411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:08.911731 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b4ff5-8486-4ad7-9c7c-70ca913d2411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:08.913483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b4ff5-8486-4ad7-9c7c-70ca913d2411 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:08.929066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 416/851] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:08.939638 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b4ff5-8486-4ad7-9c7c-70ca913d2411 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:09.046301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:09.091871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 436/852] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:08 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:29:09.093240 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbad87f-9606-45cd-86fd-e9812653d15a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.093535 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbad87f-9606-45cd-86fd-e9812653d15a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.093810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbad87f-9606-45cd-86fd-e9812653d15a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.095479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 437/853] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:09.100393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbad87f-9606-45cd-86fd-e9812653d15a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:09.123628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:09.127028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b4ff5-8486-4ad7-9c7c-70ca913d2411 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:09.183868 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa898ea6-e3ac-4758-ab25-cdd1faf41a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:09.184716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa898ea6-e3ac-4758-ab25-cdd1faf41a92 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.184716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa898ea6-e3ac-4758-ab25-cdd1faf41a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.184716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa898ea6-e3ac-4758-ab25-cdd1faf41a92 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.193067 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa898ea6-e3ac-4758-ab25-cdd1faf41a92 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:09.194528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa898ea6-e3ac-4758-ab25-cdd1faf41a92 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-aa898ea6-e3ac-4758-ab25-cdd1faf41a92 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:09.208068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:09.214430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:09.215915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:09.215915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:09.215915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:09.215915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:09.215915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:09.215915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:09.215915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:09.215915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:09.215915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:29:09.215915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:09.215915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 417/854] 10.222.0.70 () {56 vars in 1117 bytes} [Wed Aug 24 16:29:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:09.215915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b223c-911c-45f8-8912-365ceb13e681 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.215915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b223c-911c-45f8-8912-365ceb13e681 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.216903 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b223c-911c-45f8-8912-365ceb13e681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.232548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-3b0b223c-911c-45f8-8912-365ceb13e681 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:09.233587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3b0b223c-911c-45f8-8912-365ceb13e681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:09.247655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbad87f-9606-45cd-86fd-e9812653d15a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:09.289173 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1588d73-1182-44a7-b4df-ffb0aab4b6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:09.290187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1588d73-1182-44a7-b4df-ffb0aab4b6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.290672 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1588d73-1182-44a7-b4df-ffb0aab4b6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.291107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1588d73-1182-44a7-b4df-ffb0aab4b6e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.306985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1588d73-1182-44a7-b4df-ffb0aab4b6e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:09.309054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1588d73-1182-44a7-b4df-ffb0aab4b6e9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:09.330157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 418/855] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:09.331422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b223c-911c-45f8-8912-365ceb13e681 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-b1588d73-1182-44a7-b4df-ffb0aab4b6e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:09.340002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:09.341898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:09.343680 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:09.343680 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:09.343680 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:09.355296 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 438/856] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:09.357015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1588d73-1182-44a7-b4df-ffb0aab4b6e9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:09.422552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:09.444046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:09.483779 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1588d73-1182-44a7-b4df-ffb0aab4b6e9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:09.487973 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b223c-911c-45f8-8912-365ceb13e681 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:09.523876 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fd0a2-e983-435f-a959-c9dfc9626923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:09.524518 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fd0a2-e983-435f-a959-c9dfc9626923 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.524804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fd0a2-e983-435f-a959-c9dfc9626923 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.525076 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fd0a2-e983-435f-a959-c9dfc9626923 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.525720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fd0a2-e983-435f-a959-c9dfc9626923 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:09.526022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fd0a2-e983-435f-a959-c9dfc9626923 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:09.527429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fd0a2-e983-435f-a959-c9dfc9626923 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:09.529018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768b9a5-c8ef-4568-8f01-65319b41f14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:09.529898 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768b9a5-c8ef-4568-8f01-65319b41f14a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.530380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768b9a5-c8ef-4568-8f01-65319b41f14a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.530852 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768b9a5-c8ef-4568-8f01-65319b41f14a None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.531791 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768b9a5-c8ef-4568-8f01-65319b41f14a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:09.532312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768b9a5-c8ef-4568-8f01-65319b41f14a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:09.533955 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768b9a5-c8ef-4568-8f01-65319b41f14a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:09.547724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 439/857] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:09.550107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fd0a2-e983-435f-a959-c9dfc9626923 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:09.561064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 419/858] 10.222.0.70 () {56 vars in 1149 bytes} [Wed Aug 24 16:29:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:09.569416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768b9a5-c8ef-4568-8f01-65319b41f14a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:09.670745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fd0a2-e983-435f-a959-c9dfc9626923 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:09.717682 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:09.723826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768b9a5-c8ef-4568-8f01-65319b41f14a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:09.725784 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:09.727330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930a421-0f08-4be3-8095-9d92f083fbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:09.727330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930a421-0f08-4be3-8095-9d92f083fbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.727800 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930a421-0f08-4be3-8095-9d92f083fbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.727909 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930a421-0f08-4be3-8095-9d92f083fbf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.741925 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930a421-0f08-4be3-8095-9d92f083fbf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:09.743562 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930a421-0f08-4be3-8095-9d92f083fbf2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:09.756660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-a930a421-0f08-4be3-8095-9d92f083fbf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:09.756660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:09.756660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:09.756660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:09.756660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:09.756660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:09.756660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:09.756660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:09.756660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:09.756660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:09.756660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:09.756660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:09.756660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:09.756660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:09.756660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:09.758866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:09.759868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:09.759868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:09.759868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:09.759868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:09.759868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:09.759868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:09.759868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:09.759868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:09.762704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 440/859] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:09.763657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930a421-0f08-4be3-8095-9d92f083fbf2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:09.775341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcca2e-d53c-41de-ad27-52ee52a594b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:09.776063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcca2e-d53c-41de-ad27-52ee52a594b3 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.776447 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcca2e-d53c-41de-ad27-52ee52a594b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.776803 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcca2e-d53c-41de-ad27-52ee52a594b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.887683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcca2e-d53c-41de-ad27-52ee52a594b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:09.890262 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcca2e-d53c-41de-ad27-52ee52a594b3 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:09.901151 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930a421-0f08-4be3-8095-9d92f083fbf2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:09.936584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e68ef-a37b-49ae-9386-92c026656ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:09.937197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e68ef-a37b-49ae-9386-92c026656ad8 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.937474 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e68ef-a37b-49ae-9386-92c026656ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.937743 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e68ef-a37b-49ae-9386-92c026656ad8 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.938393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e68ef-a37b-49ae-9386-92c026656ad8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:09.939764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e68ef-a37b-49ae-9386-92c026656ad8 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:10.000725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 420/860] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:09 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 16:29:10.001416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcca2e-d53c-41de-ad27-52ee52a594b3 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:10.003905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:10.045279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 441/861] 10.222.0.70 () {60 vars in 1141 bytes} [Wed Aug 24 16:29:09 2022] POST /identity/v3/services => generated 251 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:10.046538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c056c2-374b-4660-8423-c967a570107a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:10.046818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c056c2-374b-4660-8423-c967a570107a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:10.047189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c056c2-374b-4660-8423-c967a570107a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:10.049523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 442/862] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:10.049523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c056c2-374b-4660-8423-c967a570107a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:10.122966 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:10.163871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcca2e-d53c-41de-ad27-52ee52a594b3 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:10.216154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0398d-1528-448e-a7e3-8f9c962fb172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:10.217150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0398d-1528-448e-a7e3-8f9c962fb172 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:10.217150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0398d-1528-448e-a7e3-8f9c962fb172 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:10.217336 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0398d-1528-448e-a7e3-8f9c962fb172 None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:10.218044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0398d-1528-448e-a7e3-8f9c962fb172 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:10.218349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0398d-1528-448e-a7e3-8f9c962fb172 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:10.219824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0398d-1528-448e-a7e3-8f9c962fb172 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:10.227093 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c056c2-374b-4660-8423-c967a570107a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:10.234891 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 421/863] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:10.236224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041a11f-e3e0-4614-8933-e8a698bb807d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:10.240885 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041a11f-e3e0-4614-8933-e8a698bb807d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:10.241718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041a11f-e3e0-4614-8933-e8a698bb807d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:10.264821 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-b041a11f-e3e0-4614-8933-e8a698bb807d None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:10.269125 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b041a11f-e3e0-4614-8933-e8a698bb807d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:10.286040 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0a02d-0e2f-4e0a-98b4-04b2545bcb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:10.286808 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0a02d-0e2f-4e0a-98b4-04b2545bcb61 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:10.287091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0a02d-0e2f-4e0a-98b4-04b2545bcb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:10.287367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0a02d-0e2f-4e0a-98b4-04b2545bcb61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:10.300237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0a02d-0e2f-4e0a-98b4-04b2545bcb61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:10.301780 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0a02d-0e2f-4e0a-98b4-04b2545bcb61 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-a2b0a02d-0e2f-4e0a-98b4-04b2545bcb61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:10.317409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:10.318418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:10.319338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:10.319338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:10.319338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:10.319338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 443/864] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:10.319564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0a02d-0e2f-4e0a-98b4-04b2545bcb61 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:10.346845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:10.362515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 422/865] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:10 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:10.363887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041a11f-e3e0-4614-8933-e8a698bb807d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:10.477265 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:10.481760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0a02d-0e2f-4e0a-98b4-04b2545bcb61 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:10.500422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041a11f-e3e0-4614-8933-e8a698bb807d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:10.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e02b59-0b13-4c5a-a4ca-a5e25f3cdc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:10.533967 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e02b59-0b13-4c5a-a4ca-a5e25f3cdc7f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:10.534943 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e02b59-0b13-4c5a-a4ca-a5e25f3cdc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:10.535751 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e02b59-0b13-4c5a-a4ca-a5e25f3cdc7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:10.537388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e02b59-0b13-4c5a-a4ca-a5e25f3cdc7f None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=b8ec0b1e5f4d434bb725297f3f867ba7) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:10.538394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e02b59-0b13-4c5a-a4ca-a5e25f3cdc7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:10.540437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e02b59-0b13-4c5a-a4ca-a5e25f3cdc7f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:10.543113 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195aa27-a740-499a-93b0-16632219da51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:10.544113 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195aa27-a740-499a-93b0-16632219da51 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:10.544113 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195aa27-a740-499a-93b0-16632219da51 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:10.544333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195aa27-a740-499a-93b0-16632219da51 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:10.545054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195aa27-a740-499a-93b0-16632219da51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:10.545363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195aa27-a740-499a-93b0-16632219da51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:10.547016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195aa27-a740-499a-93b0-16632219da51 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:10.552603 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 444/866] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:10 2022] GET /identity/v3/role_assignments?user.id=b8ec0b1e5f4d434bb725297f3f867ba7&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=e7f48343fc1542979ab4459865109e30 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:10.569583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 423/867] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:10.578448 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e02b59-0b13-4c5a-a4ca-a5e25f3cdc7f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:10.704151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195aa27-a740-499a-93b0-16632219da51 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:10.710821 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e02b59-0b13-4c5a-a4ca-a5e25f3cdc7f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:10.765086 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:10.766565 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f1048-d709-47e6-b2af-58862387e2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:10.767231 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f1048-d709-47e6-b2af-58862387e2de None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:10.767706 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f1048-d709-47e6-b2af-58862387e2de None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:10.767915 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f1048-d709-47e6-b2af-58862387e2de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:10.779222 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f1048-d709-47e6-b2af-58862387e2de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:10.780923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f1048-d709-47e6-b2af-58862387e2de None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:10.794719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-812f1048-d709-47e6-b2af-58862387e2de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:10.794719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:10.794719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:10.794719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:10.794719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:10.794719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:10.794719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:10.794719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:10.794719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:10.794719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:10.794719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:10.794719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:10.794719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:10.794719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:10.794719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:10.795654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:10.796997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:10.796997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:10.796997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:10.796997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:10.796997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:10.796997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:10.796997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:10.796997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:10.796997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 445/868] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:10.804775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f1048-d709-47e6-b2af-58862387e2de None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:10.841725 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:10.842153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195aa27-a740-499a-93b0-16632219da51 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:10.887286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a077d6-5149-45a5-91ae-a8ab4c31756f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:10.887983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a077d6-5149-45a5-91ae-a8ab4c31756f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:10.888439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a077d6-5149-45a5-91ae-a8ab4c31756f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:10.888693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a077d6-5149-45a5-91ae-a8ab4c31756f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:10.889290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a077d6-5149-45a5-91ae-a8ab4c31756f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:10.891121 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a077d6-5149-45a5-91ae-a8ab4c31756f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-85a077d6-5149-45a5-91ae-a8ab4c31756f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:10.901714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:10.902724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:10.903614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:10.903614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:10.903614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:10.903614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:10.903614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:10.903614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:10.903614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:10.903614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:10.903614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:10.903614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:10.903614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:10.903614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:10.903614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:29:10.903614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:10.903614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 424/869] 10.222.0.70 () {56 vars in 1132 bytes} [Wed Aug 24 16:29:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:10.916459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a077d6-5149-45a5-91ae-a8ab4c31756f None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:10.952213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f1048-d709-47e6-b2af-58862387e2de None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:11.002049 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01eff5-dc67-4107-96b6-0e28f2a20f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:11.002702 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01eff5-dc67-4107-96b6-0e28f2a20f38 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:11.002985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01eff5-dc67-4107-96b6-0e28f2a20f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:11.003261 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01eff5-dc67-4107-96b6-0e28f2a20f38 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:11.003966 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01eff5-dc67-4107-96b6-0e28f2a20f38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:11.004274 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01eff5-dc67-4107-96b6-0e28f2a20f38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:11.006083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01eff5-dc67-4107-96b6-0e28f2a20f38 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:11.029625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 446/870] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:11.031136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01eff5-dc67-4107-96b6-0e28f2a20f38 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:11.053940 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a077d6-5149-45a5-91ae-a8ab4c31756f None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:11.071733 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:11.095718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec2fd9-a72f-4db4-bb1c-8c5cc64105ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:11.096304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec2fd9-a72f-4db4-bb1c-8c5cc64105ec None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:11.096658 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec2fd9-a72f-4db4-bb1c-8c5cc64105ec None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:11.096936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec2fd9-a72f-4db4-bb1c-8c5cc64105ec None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:11.097569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec2fd9-a72f-4db4-bb1c-8c5cc64105ec None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:11.097874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec2fd9-a72f-4db4-bb1c-8c5cc64105ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:11.099372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec2fd9-a72f-4db4-bb1c-8c5cc64105ec None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:11.110562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 425/871] 10.222.0.70 () {56 vars in 1126 bytes} [Wed Aug 24 16:29:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:11.112032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec2fd9-a72f-4db4-bb1c-8c5cc64105ec None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:11.157142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:11.180606 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01eff5-dc67-4107-96b6-0e28f2a20f38 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:11.227453 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bd8c9-ed5c-486c-9b50-f8d33ddb2d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:11.228774 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bd8c9-ed5c-486c-9b50-f8d33ddb2d8b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:11.230181 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bd8c9-ed5c-486c-9b50-f8d33ddb2d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:11.230887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bd8c9-ed5c-486c-9b50-f8d33ddb2d8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:11.231820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bd8c9-ed5c-486c-9b50-f8d33ddb2d8b None admin] RBAC: Adding query filter params (role.id=bb34a44869834493b10a99cc751ab6c2, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=7f3899f2eb554f2fb5a7c43f660b51d5) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:11.232331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bd8c9-ed5c-486c-9b50-f8d33ddb2d8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:11.233676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec2fd9-a72f-4db4-bb1c-8c5cc64105ec None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:11.234548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bd8c9-ed5c-486c-9b50-f8d33ddb2d8b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:11.248883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 447/872] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:11 2022] GET /identity/v3/role_assignments?user.id=7f3899f2eb554f2fb5a7c43f660b51d5&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=bb34a44869834493b10a99cc751ab6c2 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:11.279620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaf565-ccb6-4df5-bbf3-131c6b69f71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:11.280259 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaf565-ccb6-4df5-bbf3-131c6b69f71c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:11.280614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaf565-ccb6-4df5-bbf3-131c6b69f71c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:11.280893 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaf565-ccb6-4df5-bbf3-131c6b69f71c None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:11.281546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeaf565-ccb6-4df5-bbf3-131c6b69f71c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:11.281852 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeaf565-ccb6-4df5-bbf3-131c6b69f71c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:11.283358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeaf565-ccb6-4df5-bbf3-131c6b69f71c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:11.289746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 426/873] 10.222.0.70 () {56 vars in 1126 bytes} [Wed Aug 24 16:29:11 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:12.320134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d153ffb-fe64-4227-98c7-5869277ac893 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:12.320698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d153ffb-fe64-4227-98c7-5869277ac893 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:12.320698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d153ffb-fe64-4227-98c7-5869277ac893 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:12.322530 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 448/874] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:12.329124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bd433-8dd0-4459-adae-f3e47ebddb72 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:12.329620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bd433-8dd0-4459-adae-f3e47ebddb72 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:12.329834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bd433-8dd0-4459-adae-f3e47ebddb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:12.349829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-730bd433-8dd0-4459-adae-f3e47ebddb72 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:12.351061 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-730bd433-8dd0-4459-adae-f3e47ebddb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:12.395858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:12.432394 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 427/875] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:12 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:12.670543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91eef0-c5f7-4976-8d5b-c9c18333e2e4 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:12.670889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91eef0-c5f7-4976-8d5b-c9c18333e2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:12.671092 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91eef0-c5f7-4976-8d5b-c9c18333e2e4 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:12.672668 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 449/876] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:12.678095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd8c81-f0da-449f-9efe-9f038a066f06 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:12.678390 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd8c81-f0da-449f-9efe-9f038a066f06 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:12.678667 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd8c81-f0da-449f-9efe-9f038a066f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:12.697889 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-05bd8c81-f0da-449f-9efe-9f038a066f06 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:12.699098 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-05bd8c81-f0da-449f-9efe-9f038a066f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:12.716639 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91eef0-c5f7-4976-8d5b-c9c18333e2e4 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:12.779581 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 428/877] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:12 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:12.839923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91eef0-c5f7-4976-8d5b-c9c18333e2e4 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:12.874625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e4a63-43d9-40eb-b6ed-87a69dfccc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:12.875255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e4a63-43d9-40eb-b6ed-87a69dfccc9d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:12.875542 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e4a63-43d9-40eb-b6ed-87a69dfccc9d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:12.875817 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e4a63-43d9-40eb-b6ed-87a69dfccc9d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:12.876427 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e4a63-43d9-40eb-b6ed-87a69dfccc9d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:12.877788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e4a63-43d9-40eb-b6ed-87a69dfccc9d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-417e4a63-43d9-40eb-b6ed-87a69dfccc9d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:12.886615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:12.887623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:12.888509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:12.888509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:12.888509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:12.888509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:12.888509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:12.888509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:12.888509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:12.888509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:12.888509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:12.888509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:12.888509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:12.888509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:12.888509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:29:12.888509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:12.888509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 450/878] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:29:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:12.894096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd8c81-f0da-449f-9efe-9f038a066f06 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:13.004169 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:13.007478 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd8c81-f0da-449f-9efe-9f038a066f06 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:13.046797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42259377-47ce-450a-a489-75a846167597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:13.047412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42259377-47ce-450a-a489-75a846167597 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.047694 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42259377-47ce-450a-a489-75a846167597 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.047970 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42259377-47ce-450a-a489-75a846167597 None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.048699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42259377-47ce-450a-a489-75a846167597 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:13.049005 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42259377-47ce-450a-a489-75a846167597 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:13.050381 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42259377-47ce-450a-a489-75a846167597 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:13.057681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e4a63-43d9-40eb-b6ed-87a69dfccc9d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:13.060146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 429/879] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:29:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:13.061896 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42259377-47ce-450a-a489-75a846167597 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:13.074082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:13.171928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42259377-47ce-450a-a489-75a846167597 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:13.175533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e4a63-43d9-40eb-b6ed-87a69dfccc9d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:13.208060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d818a-90eb-4945-ba5b-bdce712f07fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:13.208783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d818a-90eb-4945-ba5b-bdce712f07fc None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.209076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d818a-90eb-4945-ba5b-bdce712f07fc None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.209358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d818a-90eb-4945-ba5b-bdce712f07fc None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.210062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d818a-90eb-4945-ba5b-bdce712f07fc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:13.210298 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d818a-90eb-4945-ba5b-bdce712f07fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:13.211674 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d818a-90eb-4945-ba5b-bdce712f07fc None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:13.213134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461662b6-2d3d-4a67-8697-52169de7e741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:13.214458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461662b6-2d3d-4a67-8697-52169de7e741 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.214749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461662b6-2d3d-4a67-8697-52169de7e741 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.215024 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461662b6-2d3d-4a67-8697-52169de7e741 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.215616 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461662b6-2d3d-4a67-8697-52169de7e741 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:13.216998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461662b6-2d3d-4a67-8697-52169de7e741 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:13.222675 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 430/880] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:29:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:13.223929 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d818a-90eb-4945-ba5b-bdce712f07fc None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-461662b6-2d3d-4a67-8697-52169de7e741 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:13.226650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:13.229011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:13.231560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:13.231560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:13.231560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:13.231560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:13.231560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:13.231560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:13.231560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:13.231560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:13.231560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:13.231560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:13.231560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:13.231560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:13.231560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:29:13.231560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:13.241605 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 451/881] 10.222.0.70 () {56 vars in 1126 bytes} [Wed Aug 24 16:29:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:13.241605 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461662b6-2d3d-4a67-8697-52169de7e741 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:13.255372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:13.324599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:13.335530 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d818a-90eb-4945-ba5b-bdce712f07fc None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:13.352101 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461662b6-2d3d-4a67-8697-52169de7e741 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:13.373405 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6f464-be9a-4561-8eef-402712b30773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:13.374022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6f464-be9a-4561-8eef-402712b30773 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.374336 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6f464-be9a-4561-8eef-402712b30773 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.374618 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6f464-be9a-4561-8eef-402712b30773 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.375326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6f464-be9a-4561-8eef-402712b30773 None admin] RBAC: Adding query filter params (interface=public, service_id=33db1b7bd6424f3ba6c1f9edb4a72aad, region_id=RegionOne) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:13.375639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6f464-be9a-4561-8eef-402712b30773 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:13.377040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6f464-be9a-4561-8eef-402712b30773 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:13.382185 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 431/882] 10.222.0.70 () {56 vars in 1251 bytes} [Wed Aug 24 16:29:13 2022] GET /identity/v3/endpoints?service_id=33db1b7bd6424f3ba6c1f9edb4a72aad&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:13.383404 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e32143-1bef-4723-b3fe-149676518621 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.383697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e32143-1bef-4723-b3fe-149676518621 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.383972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e32143-1bef-4723-b3fe-149676518621 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.385269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 432/883] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:13.387243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62036745-385c-4add-bfe9-98ff6638a7df None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.387541 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62036745-385c-4add-bfe9-98ff6638a7df None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.387819 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62036745-385c-4add-bfe9-98ff6638a7df None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.389156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 433/884] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:13.389977 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62036745-385c-4add-bfe9-98ff6638a7df None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:13.390894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36768953-e675-4e07-afd3-a4bf028ced47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:13.391479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36768953-e675-4e07-afd3-a4bf028ced47 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.391760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36768953-e675-4e07-afd3-a4bf028ced47 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.392032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36768953-e675-4e07-afd3-a4bf028ced47 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.392710 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36768953-e675-4e07-afd3-a4bf028ced47 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:13.393008 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36768953-e675-4e07-afd3-a4bf028ced47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:13.394364 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36768953-e675-4e07-afd3-a4bf028ced47 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:13.399160 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 452/885] 10.222.0.70 () {56 vars in 1122 bytes} [Wed Aug 24 16:29:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:13.400446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e8f27-a12f-4dc0-badd-3d32b5b8d8ff None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.400832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e8f27-a12f-4dc0-badd-3d32b5b8d8ff None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.401115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e8f27-a12f-4dc0-badd-3d32b5b8d8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.419029 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f66e8f27-a12f-4dc0-badd-3d32b5b8d8ff None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:13.420230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f66e8f27-a12f-4dc0-badd-3d32b5b8d8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:13.482520 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:13.487670 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 453/886] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:13 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:13.489082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34016953-04b3-4d7d-a93e-3b56e6874ea7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.489328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34016953-04b3-4d7d-a93e-3b56e6874ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.495632 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34016953-04b3-4d7d-a93e-3b56e6874ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.496150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62036745-385c-4add-bfe9-98ff6638a7df None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:13.509717 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-34016953-04b3-4d7d-a93e-3b56e6874ea7 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:13.511393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-34016953-04b3-4d7d-a93e-3b56e6874ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:13.540040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e01ebc-d2c2-4baa-82f6-ad3126b08625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:13.540040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e01ebc-d2c2-4baa-82f6-ad3126b08625 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.540040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e01ebc-d2c2-4baa-82f6-ad3126b08625 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.540040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e01ebc-d2c2-4baa-82f6-ad3126b08625 None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.540508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e01ebc-d2c2-4baa-82f6-ad3126b08625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:13.540508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e01ebc-d2c2-4baa-82f6-ad3126b08625 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:13.546363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 434/887] 10.222.0.70 () {56 vars in 1087 bytes} [Wed Aug 24 16:29:13 2022] GET /identity/v3/services => generated 1011 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:29:13.547197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e01ebc-d2c2-4baa-82f6-ad3126b08625 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:13.612079 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:13.624987 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 454/888] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:13 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:13.683804 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e01ebc-d2c2-4baa-82f6-ad3126b08625 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:13.725943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b95808-c098-4da1-a349-bbd77fcdb9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:13.726579 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b95808-c098-4da1-a349-bbd77fcdb9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.726860 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b95808-c098-4da1-a349-bbd77fcdb9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.727138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b95808-c098-4da1-a349-bbd77fcdb9f6 None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.727866 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b95808-c098-4da1-a349-bbd77fcdb9f6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:13.728174 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b95808-c098-4da1-a349-bbd77fcdb9f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:13.729714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b95808-c098-4da1-a349-bbd77fcdb9f6 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:13.735061 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 435/889] 10.222.0.70 () {56 vars in 1122 bytes} [Wed Aug 24 16:29:13 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:13.759379 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34016953-04b3-4d7d-a93e-3b56e6874ea7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:13.889064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34016953-04b3-4d7d-a93e-3b56e6874ea7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:13.918873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b95808-c098-4da1-a349-bbd77fcdb9f6 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:13.935038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782e49d-cc6d-49fa-a9c9-6151bc189ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:13.935666 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782e49d-cc6d-49fa-a9c9-6151bc189ee6 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.935951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782e49d-cc6d-49fa-a9c9-6151bc189ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.936227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782e49d-cc6d-49fa-a9c9-6151bc189ee6 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.936951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782e49d-cc6d-49fa-a9c9-6151bc189ee6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:13.938476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782e49d-cc6d-49fa-a9c9-6151bc189ee6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:13.956321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:14.021299 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 455/890] 10.222.0.70 () {60 vars in 1141 bytes} [Wed Aug 24 16:29:13 2022] POST /identity/v3/services => generated 269 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:14.066811 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b95808-c098-4da1-a349-bbd77fcdb9f6 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:14.117668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77b162-84a4-401f-bb84-91b43c1de5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:14.118596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77b162-84a4-401f-bb84-91b43c1de5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:14.119093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77b162-84a4-401f-bb84-91b43c1de5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:14.119515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77b162-84a4-401f-bb84-91b43c1de5b6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:14.132691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77b162-84a4-401f-bb84-91b43c1de5b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-9e77b162-84a4-401f-bb84-91b43c1de5b6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:14.138283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:14.139568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:14.140961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:14.140961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:14.140961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:14.140961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:14.140961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:14.140961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:14.140961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:14.140961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:14.140961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:14.140961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:14.140961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:29:14.140961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:14.148391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 436/891] 10.222.0.70 () {56 vars in 1120 bytes} [Wed Aug 24 16:29:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:14.150363 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782e49d-cc6d-49fa-a9c9-6151bc189ee6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:14.228492 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:14.280269 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782e49d-cc6d-49fa-a9c9-6151bc189ee6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:14.324225 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459707b-37b2-4702-b84e-cbf00178fa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:14.325635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459707b-37b2-4702-b84e-cbf00178fa28 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:14.326929 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459707b-37b2-4702-b84e-cbf00178fa28 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:14.327502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459707b-37b2-4702-b84e-cbf00178fa28 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:14.328704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459707b-37b2-4702-b84e-cbf00178fa28 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:14.329386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459707b-37b2-4702-b84e-cbf00178fa28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:14.331608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459707b-37b2-4702-b84e-cbf00178fa28 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:14.356581 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 456/892] 10.222.0.70 () {56 vars in 1115 bytes} [Wed Aug 24 16:29:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:14.360304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77b162-84a4-401f-bb84-91b43c1de5b6 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:14.451134 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:14.478170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77b162-84a4-401f-bb84-91b43c1de5b6 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:14.517501 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359270e-c709-4214-b937-15e92434a1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:14.518151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359270e-c709-4214-b937-15e92434a1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:14.518436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359270e-c709-4214-b937-15e92434a1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:14.518713 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359270e-c709-4214-b937-15e92434a1c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:14.530832 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8359270e-c709-4214-b937-15e92434a1c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:14.532324 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8359270e-c709-4214-b937-15e92434a1c8 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-8359270e-c709-4214-b937-15e92434a1c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:14.549931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:14.552111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:14.553934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:14.553934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:14.553934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:14.553934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 437/893] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:14.563824 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459707b-37b2-4702-b84e-cbf00178fa28 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:14.725545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459707b-37b2-4702-b84e-cbf00178fa28 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:14.753597 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:14.769504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f52fc-a867-4a72-84cb-ba7570926444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:14.770456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f52fc-a867-4a72-84cb-ba7570926444 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:14.771190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f52fc-a867-4a72-84cb-ba7570926444 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:14.771549 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f52fc-a867-4a72-84cb-ba7570926444 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:14.772607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f52fc-a867-4a72-84cb-ba7570926444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:14.773287 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f52fc-a867-4a72-84cb-ba7570926444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:14.775839 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f52fc-a867-4a72-84cb-ba7570926444 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:14.792375 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 457/894] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:14.801958 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359270e-c709-4214-b937-15e92434a1c8 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:14.928465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359270e-c709-4214-b937-15e92434a1c8 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:14.981570 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:14.996363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08226842-e7b0-41a6-b9fc-b6a1d8fb1a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:14.997566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08226842-e7b0-41a6-b9fc-b6a1d8fb1a95 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:14.998081 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08226842-e7b0-41a6-b9fc-b6a1d8fb1a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:14.998590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08226842-e7b0-41a6-b9fc-b6a1d8fb1a95 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:15.014455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08226842-e7b0-41a6-b9fc-b6a1d8fb1a95 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:15.017950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08226842-e7b0-41a6-b9fc-b6a1d8fb1a95 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-08226842-e7b0-41a6-b9fc-b6a1d8fb1a95 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:15.041289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:15.042843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:15.043788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:15.043788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:15.043788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:15.043788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:15.043788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:15.043788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:15.043788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:15.043788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:15.043788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:29:15.043788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:15.044856 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 438/895] 10.222.0.70 () {56 vars in 1117 bytes} [Wed Aug 24 16:29:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:15.050960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f52fc-a867-4a72-84cb-ba7570926444 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:15.188356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f52fc-a867-4a72-84cb-ba7570926444 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:15.225650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6499a-bec2-4b09-bc1b-509c4d7be9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:15.226264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6499a-bec2-4b09-bc1b-509c4d7be9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:15.226543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6499a-bec2-4b09-bc1b-509c4d7be9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:15.226815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6499a-bec2-4b09-bc1b-509c4d7be9f1 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:15.227632 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6499a-bec2-4b09-bc1b-509c4d7be9f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:15.227943 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6499a-bec2-4b09-bc1b-509c4d7be9f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:15.230005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6499a-bec2-4b09-bc1b-509c4d7be9f1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:15.250463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 458/896] 10.222.0.70 () {56 vars in 1149 bytes} [Wed Aug 24 16:29:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:15.256264 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08226842-e7b0-41a6-b9fc-b6a1d8fb1a95 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:15.377273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08226842-e7b0-41a6-b9fc-b6a1d8fb1a95 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:15.418874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f1638-0f33-4a16-baef-76c26d04db5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:15.419890 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f1638-0f33-4a16-baef-76c26d04db5f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:15.420382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f1638-0f33-4a16-baef-76c26d04db5f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:15.420973 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f1638-0f33-4a16-baef-76c26d04db5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:15.487683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:15.506325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f1638-0f33-4a16-baef-76c26d04db5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:15.509030 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f1638-0f33-4a16-baef-76c26d04db5f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:15.654239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 439/897] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:15 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:29:15.660373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6499a-bec2-4b09-bc1b-509c4d7be9f1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:15.711423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:15.755981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c71f48-5671-4467-9ae7-7d8e43071dcf None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:15.756376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c71f48-5671-4467-9ae7-7d8e43071dcf None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:15.756575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c71f48-5671-4467-9ae7-7d8e43071dcf None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:15.758072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 440/898] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:15.764029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57073b72-5e8f-4e8c-851f-2cad9c95b7bd None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:15.764623 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57073b72-5e8f-4e8c-851f-2cad9c95b7bd None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:15.765146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57073b72-5e8f-4e8c-851f-2cad9c95b7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:15.782757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6499a-bec2-4b09-bc1b-509c4d7be9f1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:15.784714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-57073b72-5e8f-4e8c-851f-2cad9c95b7bd None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:15.786229 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-57073b72-5e8f-4e8c-851f-2cad9c95b7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:15.832160 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c7a8f-fedb-4413-b86b-3a359e4f2554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:15.832841 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c7a8f-fedb-4413-b86b-3a359e4f2554 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:15.833118 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c7a8f-fedb-4413-b86b-3a359e4f2554 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:15.833385 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c7a8f-fedb-4413-b86b-3a359e4f2554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:15.849068 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c7a8f-fedb-4413-b86b-3a359e4f2554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:15.850860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c7a8f-fedb-4413-b86b-3a359e4f2554 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f80c7a8f-fedb-4413-b86b-3a359e4f2554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:15.867778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:15.868843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:15.869769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:15.869769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:15.869769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:15.870004 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 441/899] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:29:15.873361 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 459/900] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:15.879147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57073b72-5e8f-4e8c-851f-2cad9c95b7bd None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:15.901014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:15.938559 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b737fc-698c-421f-8c27-1e28d4d3d8a7 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:15.938891 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b737fc-698c-421f-8c27-1e28d4d3d8a7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:15.939115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b737fc-698c-421f-8c27-1e28d4d3d8a7 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:15.940638 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 460/901] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:29:15.947369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8ca1b-9f9a-4d8c-b94f-71337c6fd201 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:15.947938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8ca1b-9f9a-4d8c-b94f-71337c6fd201 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:15.948455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8ca1b-9f9a-4d8c-b94f-71337c6fd201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:15.967449 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8aa8ca1b-9f9a-4d8c-b94f-71337c6fd201 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:15.969072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8aa8ca1b-9f9a-4d8c-b94f-71337c6fd201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:15.999485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57073b72-5e8f-4e8c-851f-2cad9c95b7bd None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:16.041081 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc54733-68cb-42cf-a69a-8dafb2590c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:16.042751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc54733-68cb-42cf-a69a-8dafb2590c8a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:16.043053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc54733-68cb-42cf-a69a-8dafb2590c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:16.043330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc54733-68cb-42cf-a69a-8dafb2590c8a None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:16.044157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc54733-68cb-42cf-a69a-8dafb2590c8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:16.044512 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc54733-68cb-42cf-a69a-8dafb2590c8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:16.046607 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc54733-68cb-42cf-a69a-8dafb2590c8a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:16.053457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 461/902] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:29:16.061438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 442/903] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:16.069551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8ca1b-9f9a-4d8c-b94f-71337c6fd201 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:16.098376 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:16.174568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc54733-68cb-42cf-a69a-8dafb2590c8a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:16.208311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8ca1b-9f9a-4d8c-b94f-71337c6fd201 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:16.253313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65640d97-9fd7-464a-8f1e-ef5c20b44689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:16.261188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65640d97-9fd7-464a-8f1e-ef5c20b44689 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:16.261188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65640d97-9fd7-464a-8f1e-ef5c20b44689 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:16.261188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65640d97-9fd7-464a-8f1e-ef5c20b44689 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:16.273318 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65640d97-9fd7-464a-8f1e-ef5c20b44689 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:16.277424 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65640d97-9fd7-464a-8f1e-ef5c20b44689 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:16.298997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-65640d97-9fd7-464a-8f1e-ef5c20b44689 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:16.298997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:16.298997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:16.298997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:16.298997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:16.298997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:16.298997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:16.298997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:16.298997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:16.298997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:16.298997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:16.298997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:16.298997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:16.298997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:16.298997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:16.300033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:16.308393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:16.308393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:16.308393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:16.308393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:16.308393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:16.308393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:16.308393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:16.308393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:16.308393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 462/904] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:16.308393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9da194-74ed-456e-bcd0-0e9fa5948654 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:16.308393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9da194-74ed-456e-bcd0-0e9fa5948654 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:16.309526 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9da194-74ed-456e-bcd0-0e9fa5948654 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:16.312883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 463/905] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:16.313286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:16.313824 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9da194-74ed-456e-bcd0-0e9fa5948654 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:16.344596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc54733-68cb-42cf-a69a-8dafb2590c8a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:16.384693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7b046-567c-49c1-9907-a7e3937f7836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:16.385347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7b046-567c-49c1-9907-a7e3937f7836 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:16.385684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7b046-567c-49c1-9907-a7e3937f7836 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:16.385969 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7b046-567c-49c1-9907-a7e3937f7836 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:16.386588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7b046-567c-49c1-9907-a7e3937f7836 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:16.387970 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7b046-567c-49c1-9907-a7e3937f7836 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-76d7b046-567c-49c1-9907-a7e3937f7836 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:16.397535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:16.398815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:16.400871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:16.400871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:16.400871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:16.400871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:16.400871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:16.400871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:16.400871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:16.400871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:16.400871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:16.400871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:16.400871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:16.400871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:16.400871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:29:16.400871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:16.400871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 443/906] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:29:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:16.402242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8887c8-d4f6-4ef2-bdb0-43a784595406 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:16.402242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8887c8-d4f6-4ef2-bdb0-43a784595406 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:16.402242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8887c8-d4f6-4ef2-bdb0-43a784595406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:16.413115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:16.421919 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-fe8887c8-d4f6-4ef2-bdb0-43a784595406 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:16.423223 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-fe8887c8-d4f6-4ef2-bdb0-43a784595406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:16.435237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9da194-74ed-456e-bcd0-0e9fa5948654 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:16.475119 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb3402-e563-4f34-ac23-20ecd0809aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:16.475733 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb3402-e563-4f34-ac23-20ecd0809aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:16.476015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb3402-e563-4f34-ac23-20ecd0809aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:16.476286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb3402-e563-4f34-ac23-20ecd0809aa3 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:16.477021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb3402-e563-4f34-ac23-20ecd0809aa3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:16.477324 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb3402-e563-4f34-ac23-20ecd0809aa3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:16.478769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb3402-e563-4f34-ac23-20ecd0809aa3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:16.491633 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 464/907] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:16.492508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb3402-e563-4f34-ac23-20ecd0809aa3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:16.499168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 444/908] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:16.500030 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8887c8-d4f6-4ef2-bdb0-43a784595406 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:16.590297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:16.624792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb3402-e563-4f34-ac23-20ecd0809aa3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:16.627829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8887c8-d4f6-4ef2-bdb0-43a784595406 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:16.663798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298cfd1-6661-48e3-bb15-40b1a4b70e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:16.664480 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298cfd1-6661-48e3-bb15-40b1a4b70e69 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:16.664722 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298cfd1-6661-48e3-bb15-40b1a4b70e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:16.665022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298cfd1-6661-48e3-bb15-40b1a4b70e69 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:16.665715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298cfd1-6661-48e3-bb15-40b1a4b70e69 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:16.667116 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298cfd1-6661-48e3-bb15-40b1a4b70e69 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:16.676215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca9881-0377-4906-a191-5469628716e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:16.677036 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca9881-0377-4906-a191-5469628716e8 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:16.677387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca9881-0377-4906-a191-5469628716e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:16.677762 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca9881-0377-4906-a191-5469628716e8 None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:16.678484 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca9881-0377-4906-a191-5469628716e8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:16.678855 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca9881-0377-4906-a191-5469628716e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:16.680389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca9881-0377-4906-a191-5469628716e8 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:16.685774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 445/909] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:29:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:16.686434 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca9881-0377-4906-a191-5469628716e8 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:16.719615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 465/910] 10.222.0.70 () {60 vars in 1141 bytes} [Wed Aug 24 16:29:16 2022] POST /identity/v3/services => generated 241 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:16.720266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298cfd1-6661-48e3-bb15-40b1a4b70e69 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:16.723761 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:16.828447 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca9881-0377-4906-a191-5469628716e8 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:16.869604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:16.872667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298cfd1-6661-48e3-bb15-40b1a4b70e69 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:16.874796 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77accb6b-655d-4d4e-8df1-d4529a8fa70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:16.875398 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77accb6b-655d-4d4e-8df1-d4529a8fa70f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:16.875678 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77accb6b-655d-4d4e-8df1-d4529a8fa70f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:16.875948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77accb6b-655d-4d4e-8df1-d4529a8fa70f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:16.876718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77accb6b-655d-4d4e-8df1-d4529a8fa70f None admin] RBAC: Adding query filter params (role.id=23f597bcb1654a0b9575d466bbb86ed6, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=7f3899f2eb554f2fb5a7c43f660b51d5) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:16.877026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77accb6b-655d-4d4e-8df1-d4529a8fa70f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:16.881515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77accb6b-655d-4d4e-8df1-d4529a8fa70f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:16.885359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 446/911] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:16 2022] GET /identity/v3/role_assignments?user.id=7f3899f2eb554f2fb5a7c43f660b51d5&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=23f597bcb1654a0b9575d466bbb86ed6 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:16.886757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77accb6b-655d-4d4e-8df1-d4529a8fa70f None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:16.921618 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d039c30-6e8b-483f-80e2-1aa3594e12da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:16.922271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d039c30-6e8b-483f-80e2-1aa3594e12da None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:16.922558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d039c30-6e8b-483f-80e2-1aa3594e12da None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:16.922837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d039c30-6e8b-483f-80e2-1aa3594e12da None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:16.923500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d039c30-6e8b-483f-80e2-1aa3594e12da None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:16.924025 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d039c30-6e8b-483f-80e2-1aa3594e12da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:16.926030 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d039c30-6e8b-483f-80e2-1aa3594e12da None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:16.934743 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 466/912] 10.222.0.70 () {56 vars in 1114 bytes} [Wed Aug 24 16:29:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:16.942398 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d039c30-6e8b-483f-80e2-1aa3594e12da None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:17.034438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:17.036761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77accb6b-655d-4d4e-8df1-d4529a8fa70f None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:17.085739 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b30df-c05a-4652-871a-02051aa1388b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:17.086382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b30df-c05a-4652-871a-02051aa1388b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:17.086663 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b30df-c05a-4652-871a-02051aa1388b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:17.086936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b30df-c05a-4652-871a-02051aa1388b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:17.087502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d039c30-6e8b-483f-80e2-1aa3594e12da None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:17.087613 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b30df-c05a-4652-871a-02051aa1388b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:17.089038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b30df-c05a-4652-871a-02051aa1388b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-027b30df-c05a-4652-871a-02051aa1388b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:17.099257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:17.100314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:17.101557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:17.101557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:17.101557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:17.101557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:17.101557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:17.101557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:17.101557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:17.101557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:17.101557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:17.101557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:17.101557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:17.101557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:17.101557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:29:17.101557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:17.101557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 447/913] 10.222.0.70 () {56 vars in 1132 bytes} [Wed Aug 24 16:29:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:17.106164 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b30df-c05a-4652-871a-02051aa1388b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:17.131150 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe4128-9e84-44af-8f6c-1d95ba18dea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:17.132160 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe4128-9e84-44af-8f6c-1d95ba18dea7 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:17.132811 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe4128-9e84-44af-8f6c-1d95ba18dea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:17.133391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe4128-9e84-44af-8f6c-1d95ba18dea7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:17.134336 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe4128-9e84-44af-8f6c-1d95ba18dea7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:17.136680 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe4128-9e84-44af-8f6c-1d95ba18dea7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:17.174896 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:17.217918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b30df-c05a-4652-871a-02051aa1388b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:17.224475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 467/914] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:29:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:17.274052 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b1bdb-71ea-4946-b7df-5723de15381b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:17.275109 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b1bdb-71ea-4946-b7df-5723de15381b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:17.275598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b1bdb-71ea-4946-b7df-5723de15381b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:17.276686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b1bdb-71ea-4946-b7df-5723de15381b None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:17.277577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b1bdb-71ea-4946-b7df-5723de15381b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:17.278204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b1bdb-71ea-4946-b7df-5723de15381b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:17.279865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b1bdb-71ea-4946-b7df-5723de15381b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:17.287304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 448/915] 10.222.0.70 () {56 vars in 1126 bytes} [Wed Aug 24 16:29:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:17.294180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe4128-9e84-44af-8f6c-1d95ba18dea7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:17.409920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe4128-9e84-44af-8f6c-1d95ba18dea7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:17.428306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:17.446597 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352bb24-9ccb-49b9-a42e-6dac069cbbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:17.447354 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352bb24-9ccb-49b9-a42e-6dac069cbbae None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:17.447726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352bb24-9ccb-49b9-a42e-6dac069cbbae None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:17.448643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352bb24-9ccb-49b9-a42e-6dac069cbbae None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:17.449526 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352bb24-9ccb-49b9-a42e-6dac069cbbae None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:17.453858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352bb24-9ccb-49b9-a42e-6dac069cbbae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:17.453858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352bb24-9ccb-49b9-a42e-6dac069cbbae None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:17.458128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 468/916] 10.222.0.70 () {56 vars in 1126 bytes} [Wed Aug 24 16:29:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:17.469186 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b1bdb-71ea-4946-b7df-5723de15381b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:17.592577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b1bdb-71ea-4946-b7df-5723de15381b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:17.615240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:17.630725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db4a6f-374a-42e4-a0b6-41945303a5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:17.631359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db4a6f-374a-42e4-a0b6-41945303a5be None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:17.631645 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db4a6f-374a-42e4-a0b6-41945303a5be None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:17.632050 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db4a6f-374a-42e4-a0b6-41945303a5be None admin] PATH_INFO: `/v3/endpoints` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:17.633067 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db4a6f-374a-42e4-a0b6-41945303a5be None admin] RBAC: Adding query filter params (interface=public, service_id=a1662a74b95540af99cb3c39cc3378d9, region_id=RegionOne) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:17.633503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db4a6f-374a-42e4-a0b6-41945303a5be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:17.635267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db4a6f-374a-42e4-a0b6-41945303a5be None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:17.644310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 449/917] 10.222.0.70 () {56 vars in 1251 bytes} [Wed Aug 24 16:29:17 2022] GET /identity/v3/endpoints?service_id=a1662a74b95540af99cb3c39cc3378d9&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:17.649797 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352bb24-9ccb-49b9-a42e-6dac069cbbae None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:17.773826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352bb24-9ccb-49b9-a42e-6dac069cbbae None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:17.818862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a4cb3-7db7-45c8-b239-f6ec0bca440e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:17.819665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a4cb3-7db7-45c8-b239-f6ec0bca440e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:17.820051 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a4cb3-7db7-45c8-b239-f6ec0bca440e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:17.820464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a4cb3-7db7-45c8-b239-f6ec0bca440e None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:17.821207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a4cb3-7db7-45c8-b239-f6ec0bca440e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:17.822711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a4cb3-7db7-45c8-b239-f6ec0bca440e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:17.831375 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 469/918] 10.222.0.70 () {56 vars in 1087 bytes} [Wed Aug 24 16:29:17 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:29:18.825616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5278f6e-5dd3-4965-9ce8-dffbb449a798 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:18.826218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5278f6e-5dd3-4965-9ce8-dffbb449a798 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:18.826218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5278f6e-5dd3-4965-9ce8-dffbb449a798 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:18.827624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 450/919] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:18.834535 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b82346-9f9e-405c-a152-4419df14b77c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:18.835478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b82346-9f9e-405c-a152-4419df14b77c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:18.835975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b82346-9f9e-405c-a152-4419df14b77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:18.857044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-83b82346-9f9e-405c-a152-4419df14b77c None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:18.858633 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-83b82346-9f9e-405c-a152-4419df14b77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:18.909575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:18.944367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 470/920] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:18.962106 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcdc10-ac19-4f6b-9d2d-c86886f0c294 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:18.962424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcdc10-ac19-4f6b-9d2d-c86886f0c294 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:18.962663 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcdc10-ac19-4f6b-9d2d-c86886f0c294 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:18.965151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 451/921] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:18.970980 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7c691-87f9-4413-a13e-8e4f1e6a28dd None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:18.971272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7c691-87f9-4413-a13e-8e4f1e6a28dd None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:18.971549 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7c691-87f9-4413-a13e-8e4f1e6a28dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:18.995482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a5b7c691-87f9-4413-a13e-8e4f1e6a28dd None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:18.997059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a5b7c691-87f9-4413-a13e-8e4f1e6a28dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:19.071319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 471/922] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:19.223979 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcdc10-ac19-4f6b-9d2d-c86886f0c294 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:19.340659 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcdc10-ac19-4f6b-9d2d-c86886f0c294 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:19.345155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7c691-87f9-4413-a13e-8e4f1e6a28dd None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:19.383704 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8765d17-7a6b-498c-8dca-c1a478890ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:19.384336 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8765d17-7a6b-498c-8dca-c1a478890ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:19.384715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8765d17-7a6b-498c-8dca-c1a478890ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:19.384899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8765d17-7a6b-498c-8dca-c1a478890ff9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:19.386097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8765d17-7a6b-498c-8dca-c1a478890ff9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:19.387456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8765d17-7a6b-498c-8dca-c1a478890ff9 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-a8765d17-7a6b-498c-8dca-c1a478890ff9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:19.396373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:19.397527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:19.398468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:19.398468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:19.398468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:19.398468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:19.398468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:19.398468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:19.398468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:19.398468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:19.398468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:19.398468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:19.398468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:19.398468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:19.398468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:29:19.398468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:19.398468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 452/923] 10.222.0.70 () {56 vars in 1126 bytes} [Wed Aug 24 16:29:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:19.399330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13b370-0882-4564-ab5f-5951f3921b68 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:19.400322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13b370-0882-4564-ab5f-5951f3921b68 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:19.400322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13b370-0882-4564-ab5f-5951f3921b68 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:19.404704 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 453/924] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:19.404704 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13b370-0882-4564-ab5f-5951f3921b68 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:19.464849 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:19.471829 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7c691-87f9-4413-a13e-8e4f1e6a28dd None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:19.481648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:19.513409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1b8d4-96e6-49fa-8945-b64885ab6de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:19.516141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1b8d4-96e6-49fa-8945-b64885ab6de8 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:19.516141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1b8d4-96e6-49fa-8945-b64885ab6de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:19.516141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1b8d4-96e6-49fa-8945-b64885ab6de8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:19.521447 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13b370-0882-4564-ab5f-5951f3921b68 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:19.527980 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1b8d4-96e6-49fa-8945-b64885ab6de8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-cac1b8d4-96e6-49fa-8945-b64885ab6de8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:19.532771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:19.533720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:19.534560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:19.534560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:19.534560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:19.534560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:19.534560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:19.534560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:19.534560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:19.534560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:19.534560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:19.534560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:19.534560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:29:19.534560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:19.540327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 472/925] 10.222.0.70 () {56 vars in 1120 bytes} [Wed Aug 24 16:29:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:19.541129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f71704-8060-407e-97bd-226cfa228a7a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:19.543012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f71704-8060-407e-97bd-226cfa228a7a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:19.543290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f71704-8060-407e-97bd-226cfa228a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:19.562144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a3f71704-8060-407e-97bd-226cfa228a7a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:19.563476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a3f71704-8060-407e-97bd-226cfa228a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:19.572013 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f31425-d75b-493f-9aed-cdd01b9df38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:19.572757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f31425-d75b-493f-9aed-cdd01b9df38a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:19.573980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f31425-d75b-493f-9aed-cdd01b9df38a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:19.574348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f31425-d75b-493f-9aed-cdd01b9df38a None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:19.575052 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f31425-d75b-493f-9aed-cdd01b9df38a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:19.575426 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f31425-d75b-493f-9aed-cdd01b9df38a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:19.576893 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f31425-d75b-493f-9aed-cdd01b9df38a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:19.582695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 454/926] 10.222.0.70 () {56 vars in 1122 bytes} [Wed Aug 24 16:29:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:19.583733 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f31425-d75b-493f-9aed-cdd01b9df38a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:19.644294 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 473/927] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:19.645546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f71704-8060-407e-97bd-226cfa228a7a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:19.718783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f31425-d75b-493f-9aed-cdd01b9df38a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:19.738355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:19.758901 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:19.761254 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ef5f6-8084-4b2c-9ed5-faaae80b828f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:19.762100 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ef5f6-8084-4b2c-9ed5-faaae80b828f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:19.762613 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ef5f6-8084-4b2c-9ed5-faaae80b828f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:19.763101 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ef5f6-8084-4b2c-9ed5-faaae80b828f None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:19.764089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ef5f6-8084-4b2c-9ed5-faaae80b828f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:19.764646 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ef5f6-8084-4b2c-9ed5-faaae80b828f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:19.765588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f71704-8060-407e-97bd-226cfa228a7a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:19.766753 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ef5f6-8084-4b2c-9ed5-faaae80b828f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:19.775732 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 455/928] 10.222.0.70 () {56 vars in 1115 bytes} [Wed Aug 24 16:29:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:19.784443 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ef5f6-8084-4b2c-9ed5-faaae80b828f None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:19.810173 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0441c0-c605-44c1-9654-f9d795896539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:19.811045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0441c0-c605-44c1-9654-f9d795896539 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:19.811322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0441c0-c605-44c1-9654-f9d795896539 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:19.811591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0441c0-c605-44c1-9654-f9d795896539 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:19.812501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0441c0-c605-44c1-9654-f9d795896539 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:19.812808 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0441c0-c605-44c1-9654-f9d795896539 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:19.814872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0441c0-c605-44c1-9654-f9d795896539 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:19.822005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 474/929] 10.222.0.70 () {56 vars in 1122 bytes} [Wed Aug 24 16:29:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:19.827654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0441c0-c605-44c1-9654-f9d795896539 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:19.892916 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ef5f6-8084-4b2c-9ed5-faaae80b828f None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:19.930439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3596510-c97d-475a-a891-14551fe511dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:19.931088 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3596510-c97d-475a-a891-14551fe511dd None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:19.931368 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3596510-c97d-475a-a891-14551fe511dd None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:19.931639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3596510-c97d-475a-a891-14551fe511dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:19.943741 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3596510-c97d-475a-a891-14551fe511dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:19.945247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3596510-c97d-475a-a891-14551fe511dd None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-b3596510-c97d-475a-a891-14551fe511dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:19.960033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:19.961670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:19.962649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:19.962649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:19.962649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:19.962649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 456/930] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:19.962649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258f2c2-4db0-44a0-94f7-bead91a1eb32 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:19.962649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258f2c2-4db0-44a0-94f7-bead91a1eb32 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:19.963026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258f2c2-4db0-44a0-94f7-bead91a1eb32 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:19.964170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 457/931] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:19.964879 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258f2c2-4db0-44a0-94f7-bead91a1eb32 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:19.970102 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0441c0-c605-44c1-9654-f9d795896539 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:19.997335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:20.014873 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa87348-4578-44bd-a89f-962a072a6ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:20.015767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa87348-4578-44bd-a89f-962a072a6ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:20.016047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa87348-4578-44bd-a89f-962a072a6ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:20.016319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa87348-4578-44bd-a89f-962a072a6ad0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:20.017057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa87348-4578-44bd-a89f-962a072a6ad0 None admin] RBAC: Adding query filter params (interface=public, service_id=3ad17823fce84d3da6fd08524f2958a6, region_id=RegionOne) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:20.017364 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa87348-4578-44bd-a89f-962a072a6ad0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:20.018924 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa87348-4578-44bd-a89f-962a072a6ad0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:20.025461 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 475/932] 10.222.0.70 () {56 vars in 1251 bytes} [Wed Aug 24 16:29:19 2022] GET /identity/v3/endpoints?service_id=3ad17823fce84d3da6fd08524f2958a6&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:20.026677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa87348-4578-44bd-a89f-962a072a6ad0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:20.054045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:20.085377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258f2c2-4db0-44a0-94f7-bead91a1eb32 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:20.122956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8359d-997d-408b-a90d-67abfb9bf263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:20.123575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8359d-997d-408b-a90d-67abfb9bf263 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:20.123858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8359d-997d-408b-a90d-67abfb9bf263 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:20.124129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8359d-997d-408b-a90d-67abfb9bf263 None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:20.124858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8359d-997d-408b-a90d-67abfb9bf263 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:20.126313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8359d-997d-408b-a90d-67abfb9bf263 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:20.129037 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: WARNING keystone.server.flask.common [None req-6ca8359d-997d-408b-a90d-67abfb9bf263 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 16:29:20.149410 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa87348-4578-44bd-a89f-962a072a6ad0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:20.186559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.sql.core [None req-6ca8359d-997d-408b-a90d-67abfb9bf263 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 16:29:20.186559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 16:29:20.186559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [parameters: {'id': 'd114475a18ec4554ab8f9eee778f083f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 16:29:20.186559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81844) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 16:29:20.190382 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e99e90-92a3-41e5-aa33-7ec0f9406ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:20.191002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e99e90-92a3-41e5-aa33-7ec0f9406ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:20.191280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e99e90-92a3-41e5-aa33-7ec0f9406ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:20.191555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e99e90-92a3-41e5-aa33-7ec0f9406ca0 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:20.192213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e99e90-92a3-41e5-aa33-7ec0f9406ca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:20.192550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e99e90-92a3-41e5-aa33-7ec0f9406ca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:20.193596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-6ca8359d-997d-408b-a90d-67abfb9bf263 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d114475a18ec4554ab8f9eee778f083f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d114475a18ec4554ab8f9eee778f083f. Aug 24 16:29:20.193596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:20.193596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:29:20.193596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:29:20.193596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:29:20.193596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:29:20.193596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:29:20.193596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:29:20.193596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:29:20.193596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:29:20.193596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:29:20.193596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:29:20.193596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application result.read() Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 16:29:20.195187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application yield resource Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise self.value Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application session.commit() Aug 24 16:29:20.198054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self.session.flush() Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application compat.raise_( Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 16:29:20.200801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 16:29:20.203076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 16:29:20.203076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 16:29:20.203076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 16:29:20.203076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 16:29:20.203076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 16:29:20.203076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 16:29:20.203076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 16:29:20.203076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 16:29:20.203076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 16:29:20.203076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 16:29:20.203076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 16:29:20.203076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 16:29:20.203076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 16:29:20.203076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 16:29:20.203076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application result.read() Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:29:20.206116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:29:20.207679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:29:20.207679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:29:20.207679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:29:20.207679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:29:20.207679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:29:20.207679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:29:20.207679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 16:29:20.207679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 16:29:20.207679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [parameters: {'id': 'd114475a18ec4554ab8f9eee778f083f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 16:29:20.207679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 16:29:20.207679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:20.207679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:20.207679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:20.207679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:20.207679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:20.208947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:20.210566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:20.210566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 16:29:20.210566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 16:29:20.210566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:20.210566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:20.210566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 16:29:20.210566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 16:29:20.210566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d114475a18ec4554ab8f9eee778f083f. Aug 24 16:29:20.210566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:20.211725 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e99e90-92a3-41e5-aa33-7ec0f9406ca0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:20.211972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 458/933] 10.222.0.70 () {60 vars in 1140 bytes} [Wed Aug 24 16:29:19 2022] POST /identity/v3/projects => generated 262 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:20.211972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f4f3e-562e-4c01-a3f5-dc71ed1ec854 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:20.211972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f4f3e-562e-4c01-a3f5-dc71ed1ec854 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:20.211972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f4f3e-562e-4c01-a3f5-dc71ed1ec854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:20.212664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 476/934] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:20.212664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e99e90-92a3-41e5-aa33-7ec0f9406ca0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:20.222964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-c79f4f3e-562e-4c01-a3f5-dc71ed1ec854 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:20.224574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c79f4f3e-562e-4c01-a3f5-dc71ed1ec854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:20.313492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 459/935] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:20.313492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f4f3e-562e-4c01-a3f5-dc71ed1ec854 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:20.322074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:20.338328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e99e90-92a3-41e5-aa33-7ec0f9406ca0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:20.345355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:20.378436 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6d6a3-b3d2-4de8-9b0b-905f8714ea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:20.379051 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6d6a3-b3d2-4de8-9b0b-905f8714ea1b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:20.379360 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6d6a3-b3d2-4de8-9b0b-905f8714ea1b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:20.379685 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6d6a3-b3d2-4de8-9b0b-905f8714ea1b None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:20.380279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6d6a3-b3d2-4de8-9b0b-905f8714ea1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:20.381693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6d6a3-b3d2-4de8-9b0b-905f8714ea1b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:20.400457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 477/936] 10.222.0.70 () {56 vars in 1087 bytes} [Wed Aug 24 16:29:20 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:29:20.400457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6d6a3-b3d2-4de8-9b0b-905f8714ea1b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:20.497857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f4f3e-562e-4c01-a3f5-dc71ed1ec854 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:20.579377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9919b9c-70ff-47b0-9d18-8d393c4f8cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:20.579959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9919b9c-70ff-47b0-9d18-8d393c4f8cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:20.580243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9919b9c-70ff-47b0-9d18-8d393c4f8cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:20.580583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9919b9c-70ff-47b0-9d18-8d393c4f8cc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:20.588602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9919b9c-70ff-47b0-9d18-8d393c4f8cc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:20.590354 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9919b9c-70ff-47b0-9d18-8d393c4f8cc8 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:20.605437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-b9919b9c-70ff-47b0-9d18-8d393c4f8cc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:20.605437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:20.605437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:20.605437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:20.605437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:20.605437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:20.605437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:20.605437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:20.605437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:20.605437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:20.605437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:20.605437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:20.605437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:20.605437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:20.605437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:20.606429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:20.607334 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:20.607334 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:20.607334 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:20.607334 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:20.607334 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:20.607334 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:20.607334 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:20.607334 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:20.607794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6d6a3-b3d2-4de8-9b0b-905f8714ea1b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:20.609707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 460/937] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:20.610664 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9919b9c-70ff-47b0-9d18-8d393c4f8cc8 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:20.654121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5285ba-5636-4250-a41b-5d76ac2003ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:20.654922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5285ba-5636-4250-a41b-5d76ac2003ae None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:20.655659 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5285ba-5636-4250-a41b-5d76ac2003ae None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:20.655659 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5285ba-5636-4250-a41b-5d76ac2003ae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:20.664737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5285ba-5636-4250-a41b-5d76ac2003ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:20.666244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5285ba-5636-4250-a41b-5d76ac2003ae None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-fd5285ba-5636-4250-a41b-5d76ac2003ae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:20.679459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:20.681387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:20.685002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:20.685002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:20.685002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:20.685002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:20.685002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:20.685002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:20.685002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:20.685002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:20.685002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:29:20.685002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:20.685002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 478/938] 10.222.0.70 () {56 vars in 1117 bytes} [Wed Aug 24 16:29:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:20.685002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5285ba-5636-4250-a41b-5d76ac2003ae None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:20.696040 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:20.729655 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:20.764608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9919b9c-70ff-47b0-9d18-8d393c4f8cc8 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:20.805696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a74ad-e8e9-4887-be46-2a17fdb7428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:20.806500 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a74ad-e8e9-4887-be46-2a17fdb7428b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:20.806780 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a74ad-e8e9-4887-be46-2a17fdb7428b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:20.807055 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a74ad-e8e9-4887-be46-2a17fdb7428b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:20.808253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a74ad-e8e9-4887-be46-2a17fdb7428b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:20.809714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a74ad-e8e9-4887-be46-2a17fdb7428b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-f32a74ad-e8e9-4887-be46-2a17fdb7428b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:20.819063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:20.820087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:20.821373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:20.821373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:20.821373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:20.821373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:20.821373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:20.821373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:20.821373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:20.821373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:20.821373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:20.821373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:20.821373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:20.821373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:20.821373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:29:20.821373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:20.821373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 461/939] 10.222.0.70 () {56 vars in 1132 bytes} [Wed Aug 24 16:29:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:20.822557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a74ad-e8e9-4887-be46-2a17fdb7428b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:20.844169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5285ba-5636-4250-a41b-5d76ac2003ae None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:20.899350 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16808030-397d-4c89-bf3e-dc44cfdee0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:20.900625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808030-397d-4c89-bf3e-dc44cfdee0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:20.900952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808030-397d-4c89-bf3e-dc44cfdee0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:20.901230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808030-397d-4c89-bf3e-dc44cfdee0e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:20.901925 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16808030-397d-4c89-bf3e-dc44cfdee0e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:20.902420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16808030-397d-4c89-bf3e-dc44cfdee0e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:20.903890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16808030-397d-4c89-bf3e-dc44cfdee0e5 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:20.932692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 479/940] 10.222.0.70 () {56 vars in 1112 bytes} [Wed Aug 24 16:29:20 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:20.932692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16808030-397d-4c89-bf3e-dc44cfdee0e5 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:20.969011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a74ad-e8e9-4887-be46-2a17fdb7428b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:21.048512 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:21.059337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4b1c1-a837-4ad3-984f-b701cb45eb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:21.061354 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4b1c1-a837-4ad3-984f-b701cb45eb5c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:21.065596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4b1c1-a837-4ad3-984f-b701cb45eb5c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:21.065596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4b1c1-a837-4ad3-984f-b701cb45eb5c None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:21.065596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4b1c1-a837-4ad3-984f-b701cb45eb5c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:21.065596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4b1c1-a837-4ad3-984f-b701cb45eb5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:21.068313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4b1c1-a837-4ad3-984f-b701cb45eb5c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:21.075338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:21.097503 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16808030-397d-4c89-bf3e-dc44cfdee0e5 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:21.120306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 462/941] 10.222.0.70 () {56 vars in 1149 bytes} [Wed Aug 24 16:29:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:21.128883 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4b1c1-a837-4ad3-984f-b701cb45eb5c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:21.142586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec13c97-c07d-46e0-b014-87dc4c92e0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:21.143217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec13c97-c07d-46e0-b014-87dc4c92e0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:21.143502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec13c97-c07d-46e0-b014-87dc4c92e0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:21.143778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec13c97-c07d-46e0-b014-87dc4c92e0a7 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:21.144487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec13c97-c07d-46e0-b014-87dc4c92e0a7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:21.144773 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec13c97-c07d-46e0-b014-87dc4c92e0a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:21.146186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec13c97-c07d-46e0-b014-87dc4c92e0a7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:21.153044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 480/942] 10.222.0.70 () {56 vars in 1126 bytes} [Wed Aug 24 16:29:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:21.163165 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec13c97-c07d-46e0-b014-87dc4c92e0a7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:21.291974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4b1c1-a837-4ad3-984f-b701cb45eb5c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:21.332271 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84d2cb-ae5c-475a-b354-3972aa58b0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:21.332991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84d2cb-ae5c-475a-b354-3972aa58b0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:21.333276 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84d2cb-ae5c-475a-b354-3972aa58b0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:21.333623 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84d2cb-ae5c-475a-b354-3972aa58b0d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:21.340074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec13c97-c07d-46e0-b014-87dc4c92e0a7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:21.346490 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84d2cb-ae5c-475a-b354-3972aa58b0d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:21.348048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84d2cb-ae5c-475a-b354-3972aa58b0d7 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-5f84d2cb-ae5c-475a-b354-3972aa58b0d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:21.364793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:21.365798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:21.367037 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:21.367037 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:21.367037 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:21.367037 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 463/943] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:21.373402 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84d2cb-ae5c-475a-b354-3972aa58b0d7 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:21.397977 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:21.406699 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568dd90-4586-48cc-98c2-4ab5451e8054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:21.407352 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568dd90-4586-48cc-98c2-4ab5451e8054 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:21.407723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568dd90-4586-48cc-98c2-4ab5451e8054 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:21.408156 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568dd90-4586-48cc-98c2-4ab5451e8054 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:21.409400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568dd90-4586-48cc-98c2-4ab5451e8054 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:21.410307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568dd90-4586-48cc-98c2-4ab5451e8054 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:21.412471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568dd90-4586-48cc-98c2-4ab5451e8054 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:21.454686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 481/944] 10.222.0.70 () {56 vars in 1126 bytes} [Wed Aug 24 16:29:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:21.454686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568dd90-4586-48cc-98c2-4ab5451e8054 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:21.460436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:21.531017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84d2cb-ae5c-475a-b354-3972aa58b0d7 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:21.581041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51369a3f-8f49-4241-8f95-5c5b60702522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:21.581673 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51369a3f-8f49-4241-8f95-5c5b60702522 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:21.581958 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51369a3f-8f49-4241-8f95-5c5b60702522 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:21.582235 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51369a3f-8f49-4241-8f95-5c5b60702522 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:21.582900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51369a3f-8f49-4241-8f95-5c5b60702522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:21.583206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51369a3f-8f49-4241-8f95-5c5b60702522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:21.584720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51369a3f-8f49-4241-8f95-5c5b60702522 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:21.600503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 464/945] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:21.610158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51369a3f-8f49-4241-8f95-5c5b60702522 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:21.632840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568dd90-4586-48cc-98c2-4ab5451e8054 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:21.697666 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415216a-2d14-484f-83b4-c6cf69161a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:21.699676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415216a-2d14-484f-83b4-c6cf69161a03 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:21.700193 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415216a-2d14-484f-83b4-c6cf69161a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:21.700765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415216a-2d14-484f-83b4-c6cf69161a03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:21.701726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415216a-2d14-484f-83b4-c6cf69161a03 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:21.703873 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415216a-2d14-484f-83b4-c6cf69161a03 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:21.749322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51369a3f-8f49-4241-8f95-5c5b60702522 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:21.763310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:21.819827 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b52ca-d4ba-48d9-b31f-fa937a6a2f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:21.821254 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b52ca-d4ba-48d9-b31f-fa937a6a2f11 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:21.821748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b52ca-d4ba-48d9-b31f-fa937a6a2f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:21.822248 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b52ca-d4ba-48d9-b31f-fa937a6a2f11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:21.834126 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b52ca-d4ba-48d9-b31f-fa937a6a2f11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:21.834800 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 482/946] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:29:21 2022] POST /identity/v3/endpoints => generated 347 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:21.835886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b52ca-d4ba-48d9-b31f-fa937a6a2f11 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:21.849999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-6b8b52ca-d4ba-48d9-b31f-fa937a6a2f11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:21.849999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:21.849999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:21.849999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:21.849999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:21.849999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:21.849999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:21.849999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:21.849999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:21.849999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:21.849999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:21.849999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:21.849999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:21.849999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:21.849999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:21.851054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:21.852418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:21.852418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:21.852418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:21.852418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:21.852418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:21.852418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:21.852418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:21.852418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:21.852418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 465/947] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:21.859579 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415216a-2d14-484f-83b4-c6cf69161a03 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:21.928174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:21.992372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415216a-2d14-484f-83b4-c6cf69161a03 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:22.033998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe1bae-51a9-407e-9d30-f585672c38f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:22.035293 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe1bae-51a9-407e-9d30-f585672c38f4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:22.035894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe1bae-51a9-407e-9d30-f585672c38f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:22.036578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe1bae-51a9-407e-9d30-f585672c38f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:22.037945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe1bae-51a9-407e-9d30-f585672c38f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:22.038608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe1bae-51a9-407e-9d30-f585672c38f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:22.040988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe1bae-51a9-407e-9d30-f585672c38f4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:22.068549 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 483/948] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:22.072097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b52ca-d4ba-48d9-b31f-fa937a6a2f11 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:22.209617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b52ca-d4ba-48d9-b31f-fa937a6a2f11 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:22.250869 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85ff99-9a18-49d5-bb9b-bac815871357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:22.251564 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85ff99-9a18-49d5-bb9b-bac815871357 None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:22.251850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85ff99-9a18-49d5-bb9b-bac815871357 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:22.252143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85ff99-9a18-49d5-bb9b-bac815871357 None admin] PATH_INFO: `/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/7f3899f2eb554f2fb5a7c43f660b51d5/roles/23f597bcb1654a0b9575d466bbb86ed6` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:22.254545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85ff99-9a18-49d5-bb9b-bac815871357 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc920dce30734ac7a69dc039d8c74ce1, user_id=7f3899f2eb554f2fb5a7c43f660b51d5, role_id=23f597bcb1654a0b9575d466bbb86ed6)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:22.256045 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85ff99-9a18-49d5-bb9b-bac815871357 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:22.376820 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 466/949] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:29:22 2022] PUT /identity/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/7f3899f2eb554f2fb5a7c43f660b51d5/roles/23f597bcb1654a0b9575d466bbb86ed6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:29:22.840659 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d996a77-c6a9-42ba-9e3a-8c02958a613c None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:22.841616 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d996a77-c6a9-42ba-9e3a-8c02958a613c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:22.842384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d996a77-c6a9-42ba-9e3a-8c02958a613c None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:22.845715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 484/950] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:22.861721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d138d-9e5f-4fd6-b73a-6d31635834e7 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:22.862966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d138d-9e5f-4fd6-b73a-6d31635834e7 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:22.864502 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d138d-9e5f-4fd6-b73a-6d31635834e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:22.904028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-bd5d138d-9e5f-4fd6-b73a-6d31635834e7 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:22.906834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-bd5d138d-9e5f-4fd6-b73a-6d31635834e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:22.946229 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:23.034858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 467/951] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:29:23.185909 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02635dfc-7346-4f2c-9c1e-7f6d03ac3156 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:23.186482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02635dfc-7346-4f2c-9c1e-7f6d03ac3156 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:23.186482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02635dfc-7346-4f2c-9c1e-7f6d03ac3156 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:23.188122 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 485/952] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:23.194974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efb1d2-2656-4833-bb65-8bead95619b7 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:23.195494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efb1d2-2656-4833-bb65-8bead95619b7 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:23.195881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efb1d2-2656-4833-bb65-8bead95619b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:23.227273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-23efb1d2-2656-4833-bb65-8bead95619b7 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:23.228528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-23efb1d2-2656-4833-bb65-8bead95619b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:23.310213 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 468/953] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:23.521963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02635dfc-7346-4f2c-9c1e-7f6d03ac3156 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:23.586895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efb1d2-2656-4833-bb65-8bead95619b7 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:23.679696 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02635dfc-7346-4f2c-9c1e-7f6d03ac3156 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:23.695861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:23.707696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:23.726642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efb1d2-2656-4833-bb65-8bead95619b7 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:23.735838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0723b70-cb4f-4e28-a84f-502781f99500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:23.736552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0723b70-cb4f-4e28-a84f-502781f99500 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:23.736840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0723b70-cb4f-4e28-a84f-502781f99500 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:23.737181 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0723b70-cb4f-4e28-a84f-502781f99500 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:23.737785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0723b70-cb4f-4e28-a84f-502781f99500 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:23.739685 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0723b70-cb4f-4e28-a84f-502781f99500 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-d0723b70-cb4f-4e28-a84f-502781f99500 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:23.752244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:23.753643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:23.754544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:23.754544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:23.754544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:23.754544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:23.754544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:23.754544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:23.754544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:23.754544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:23.754544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:23.754544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:23.754544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:23.754544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:23.754544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:29:23.754544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:23.754544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 486/954] 10.222.0.70 () {56 vars in 1126 bytes} [Wed Aug 24 16:29:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:23.761966 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0723b70-cb4f-4e28-a84f-502781f99500 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:23.773958 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a98f0e-5fc2-4b8b-9ec3-932d6dcdd5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:23.774926 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a98f0e-5fc2-4b8b-9ec3-932d6dcdd5a2 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:23.775424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a98f0e-5fc2-4b8b-9ec3-932d6dcdd5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:23.776617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a98f0e-5fc2-4b8b-9ec3-932d6dcdd5a2 None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:23.777549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a98f0e-5fc2-4b8b-9ec3-932d6dcdd5a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:23.779230 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a98f0e-5fc2-4b8b-9ec3-932d6dcdd5a2 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:23.839614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.sql.core [None req-03a98f0e-5fc2-4b8b-9ec3-932d6dcdd5a2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 16:29:23.839614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 16:29:23.839614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [parameters: {'user_id': '9d5354b07ac345a888c0d66f6d932170', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 16:29:23.839614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81844) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 16:29:23.847017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-03a98f0e-5fc2-4b8b-9ec3-932d6dcdd5a2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 16:29:23.847017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:23.847017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:29:23.847017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:29:23.847017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:29:23.847017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:29:23.847017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:29:23.847017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:29:23.847017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:29:23.847017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:29:23.847017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:29:23.847017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:29:23.847017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:29:23.847017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application result.read() Aug 24 16:29:23.847017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:29:23.851038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application yield resource Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise self.value Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application session.commit() Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 16:29:23.852914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self.session.flush() Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application compat.raise_( Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 16:29:23.857974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:29:23.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application result.read() Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:29:23.864880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:29:23.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:29:23.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:29:23.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:29:23.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:29:23.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 16:29:23.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 16:29:23.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [parameters: {'user_id': '9d5354b07ac345a888c0d66f6d932170', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 16:29:23.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 16:29:23.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:23.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:23.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:23.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:23.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:23.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:23.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 16:29:23.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 16:29:23.877813 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 16:29:23.877813 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 16:29:23.877813 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 16:29:23.877813 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 16:29:23.877813 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 16:29:23.877813 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:23.877813 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 469/955] 10.222.0.70 () {60 vars in 1131 bytes} [Wed Aug 24 16:29:23 2022] POST /identity/v3/users => generated 160 bytes in 262 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:23.877813 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a98f0e-5fc2-4b8b-9ec3-932d6dcdd5a2 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:23.927605 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0723b70-cb4f-4e28-a84f-502781f99500 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:24.000354 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a98f0e-5fc2-4b8b-9ec3-932d6dcdd5a2 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:24.011134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e121148-edd9-420f-9da0-0fe77cda060c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:24.013115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e121148-edd9-420f-9da0-0fe77cda060c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:24.013757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e121148-edd9-420f-9da0-0fe77cda060c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:24.014398 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e121148-edd9-420f-9da0-0fe77cda060c None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:24.016499 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e121148-edd9-420f-9da0-0fe77cda060c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:24.017224 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e121148-edd9-420f-9da0-0fe77cda060c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:24.019279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e121148-edd9-420f-9da0-0fe77cda060c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:24.035467 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 487/956] 10.222.0.70 () {56 vars in 1122 bytes} [Wed Aug 24 16:29:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:24.043019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e121148-edd9-420f-9da0-0fe77cda060c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:24.054440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a85871-2d12-4d19-9107-749420f134bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:24.055115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a85871-2d12-4d19-9107-749420f134bb None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:24.055415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a85871-2d12-4d19-9107-749420f134bb None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:24.055683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a85871-2d12-4d19-9107-749420f134bb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:24.067324 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a85871-2d12-4d19-9107-749420f134bb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:24.069705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a85871-2d12-4d19-9107-749420f134bb None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-48a85871-2d12-4d19-9107-749420f134bb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:24.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:24.084882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:24.087432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:24.087432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:24.087432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:24.087432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:24.087432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:24.087432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:24.087432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:24.087432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:24.087432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:29:24.087432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:24.087432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 470/957] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:29:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:24.093228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a85871-2d12-4d19-9107-749420f134bb None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:24.112825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:24.129850 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:24.259558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e121148-edd9-420f-9da0-0fe77cda060c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:24.273128 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a85871-2d12-4d19-9107-749420f134bb None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:24.317866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012ad7c-9a87-4d94-a99d-ca0704fdf538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:24.318558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012ad7c-9a87-4d94-a99d-ca0704fdf538 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:24.318830 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012ad7c-9a87-4d94-a99d-ca0704fdf538 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:24.319191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012ad7c-9a87-4d94-a99d-ca0704fdf538 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:24.320059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012ad7c-9a87-4d94-a99d-ca0704fdf538 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:24.320227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012ad7c-9a87-4d94-a99d-ca0704fdf538 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:24.322526 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012ad7c-9a87-4d94-a99d-ca0704fdf538 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:24.338857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 488/958] 10.222.0.70 () {56 vars in 1122 bytes} [Wed Aug 24 16:29:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:24.338857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c909b-4c89-4d74-99af-6f531aaedddf None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:24.338857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c909b-4c89-4d74-99af-6f531aaedddf None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:24.338857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c909b-4c89-4d74-99af-6f531aaedddf None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:24.338857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 489/959] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:24.342852 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d144e-0cb3-4a01-9642-7d64beb93bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:24.343649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d144e-0cb3-4a01-9642-7d64beb93bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:24.344019 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d144e-0cb3-4a01-9642-7d64beb93bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:24.344501 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d144e-0cb3-4a01-9642-7d64beb93bf3 None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:24.345160 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d144e-0cb3-4a01-9642-7d64beb93bf3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:24.345466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d144e-0cb3-4a01-9642-7d64beb93bf3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:24.346624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c909b-4c89-4d74-99af-6f531aaedddf None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:24.346935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d144e-0cb3-4a01-9642-7d64beb93bf3 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:24.399686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 471/960] 10.222.0.70 () {56 vars in 1101 bytes} [Wed Aug 24 16:29:24 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:24.400988 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533598b-efd0-48af-a744-a87591e0065d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:24.401774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533598b-efd0-48af-a744-a87591e0065d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:24.402068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533598b-efd0-48af-a744-a87591e0065d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:24.438408 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-e533598b-efd0-48af-a744-a87591e0065d None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:24.441559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e533598b-efd0-48af-a744-a87591e0065d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:24.564435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:24.575185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c909b-4c89-4d74-99af-6f531aaedddf None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:24.582240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:24.598155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 472/961] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:24.638797 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f55cea-39d6-49f2-92f6-a5280423fecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:24.640001 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f55cea-39d6-49f2-92f6-a5280423fecd None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:24.640929 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f55cea-39d6-49f2-92f6-a5280423fecd None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:24.641495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f55cea-39d6-49f2-92f6-a5280423fecd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:24.642860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f55cea-39d6-49f2-92f6-a5280423fecd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:24.648472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f55cea-39d6-49f2-92f6-a5280423fecd None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:24.771761 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 490/962] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:29:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:24.801112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c61539-5c3d-44c1-9565-10bfdf5518f2 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:24.801472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c61539-5c3d-44c1-9565-10bfdf5518f2 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:24.801747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c61539-5c3d-44c1-9565-10bfdf5518f2 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:24.803643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 473/963] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:24.810332 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccec22-5074-44b0-896f-659ab1766ff7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:24.810650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccec22-5074-44b0-896f-659ab1766ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:24.810965 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccec22-5074-44b0-896f-659ab1766ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:24.837600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ddccec22-5074-44b0-896f-659ab1766ff7 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:24.838791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ddccec22-5074-44b0-896f-659ab1766ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:24.887359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c61539-5c3d-44c1-9565-10bfdf5518f2 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:24.945714 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 491/964] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:25.061931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c61539-5c3d-44c1-9565-10bfdf5518f2 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:25.127439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08551e-f8c5-4ef2-9042-33290a4162b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:25.128594 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08551e-f8c5-4ef2-9042-33290a4162b6 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:25.129015 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08551e-f8c5-4ef2-9042-33290a4162b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:25.129577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08551e-f8c5-4ef2-9042-33290a4162b6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:25.130811 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08551e-f8c5-4ef2-9042-33290a4162b6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:25.133157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08551e-f8c5-4ef2-9042-33290a4162b6 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-bc08551e-f8c5-4ef2-9042-33290a4162b6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:25.150343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:25.151886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:25.157666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:25.157666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:25.157666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:25.157666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:25.157666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:25.157666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:25.157666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:25.157666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:25.157666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:25.157666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:25.157666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:25.157666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:25.157666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:29:25.157666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:25.157666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 474/965] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:25.161108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddccec22-5074-44b0-896f-659ab1766ff7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:25.285544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:25.288590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08551e-f8c5-4ef2-9042-33290a4162b6 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:25.308154 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddccec22-5074-44b0-896f-659ab1766ff7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:25.357199 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033a9ab-195a-42e8-ba68-3f13e6ffd358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:25.358173 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033a9ab-195a-42e8-ba68-3f13e6ffd358 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:25.358608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033a9ab-195a-42e8-ba68-3f13e6ffd358 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:25.359082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033a9ab-195a-42e8-ba68-3f13e6ffd358 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:25.360133 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033a9ab-195a-42e8-ba68-3f13e6ffd358 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:25.360621 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033a9ab-195a-42e8-ba68-3f13e6ffd358 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:25.363009 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033a9ab-195a-42e8-ba68-3f13e6ffd358 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:25.377250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 492/966] 10.222.0.70 () {56 vars in 1112 bytes} [Wed Aug 24 16:29:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:25.381497 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033a9ab-195a-42e8-ba68-3f13e6ffd358 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:25.395733 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:25.479982 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08551e-f8c5-4ef2-9042-33290a4162b6 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:25.543712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b796bbf-ce67-421a-b2bf-c7b30b04fbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:25.545615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b796bbf-ce67-421a-b2bf-c7b30b04fbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:25.546079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b796bbf-ce67-421a-b2bf-c7b30b04fbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:25.546530 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b796bbf-ce67-421a-b2bf-c7b30b04fbb1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:25.564975 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b796bbf-ce67-421a-b2bf-c7b30b04fbb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-1b796bbf-ce67-421a-b2bf-c7b30b04fbb1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:25.573506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:25.574873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:25.577389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:25.577389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:25.577389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:25.577389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:25.577389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:25.577389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:25.577389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:25.577389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:25.577389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:25.577389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:25.577389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:29:25.577389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:25.577389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 475/967] 10.222.0.70 () {56 vars in 1120 bytes} [Wed Aug 24 16:29:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:25.583531 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b796bbf-ce67-421a-b2bf-c7b30b04fbb1 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:25.595547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033a9ab-195a-42e8-ba68-3f13e6ffd358 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:25.660122 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e51fe-327a-4a93-918a-2470ea93b86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:25.660712 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e51fe-327a-4a93-918a-2470ea93b86e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:25.660984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e51fe-327a-4a93-918a-2470ea93b86e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:25.661259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e51fe-327a-4a93-918a-2470ea93b86e None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:25.661904 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e51fe-327a-4a93-918a-2470ea93b86e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:25.662211 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e51fe-327a-4a93-918a-2470ea93b86e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:25.663590 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e51fe-327a-4a93-918a-2470ea93b86e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:25.676439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 493/968] 10.222.0.70 () {56 vars in 1112 bytes} [Wed Aug 24 16:29:25 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:25.742955 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b796bbf-ce67-421a-b2bf-c7b30b04fbb1 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:25.781705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:25.805428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447666c8-4a85-455d-9b5c-8662eadd5195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:25.806443 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447666c8-4a85-455d-9b5c-8662eadd5195 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:25.807104 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447666c8-4a85-455d-9b5c-8662eadd5195 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:25.807483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447666c8-4a85-455d-9b5c-8662eadd5195 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:25.808717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447666c8-4a85-455d-9b5c-8662eadd5195 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:25.809230 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447666c8-4a85-455d-9b5c-8662eadd5195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:25.811682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447666c8-4a85-455d-9b5c-8662eadd5195 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:25.826418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 476/969] 10.222.0.70 () {56 vars in 1115 bytes} [Wed Aug 24 16:29:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:25.833847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e51fe-327a-4a93-918a-2470ea93b86e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:25.860331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:25.972863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e51fe-327a-4a93-918a-2470ea93b86e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:26.022488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec0ba46-fe95-4410-8d0c-f83d93a8af0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:26.023443 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0ba46-fe95-4410-8d0c-f83d93a8af0c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:26.023916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0ba46-fe95-4410-8d0c-f83d93a8af0c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:26.024437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0ba46-fe95-4410-8d0c-f83d93a8af0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:26.041817 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0ba46-fe95-4410-8d0c-f83d93a8af0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:26.044088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0ba46-fe95-4410-8d0c-f83d93a8af0c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-0ec0ba46-fe95-4410-8d0c-f83d93a8af0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:26.068259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:26.070047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:26.071901 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:26.071901 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:26.071901 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:26.071901 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 494/970] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:26.082102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447666c8-4a85-455d-9b5c-8662eadd5195 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:26.234031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447666c8-4a85-455d-9b5c-8662eadd5195 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:26.281707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d069073-e3fe-4c50-aa18-5898937a40c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:26.282486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d069073-e3fe-4c50-aa18-5898937a40c8 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:26.282945 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d069073-e3fe-4c50-aa18-5898937a40c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:26.283342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d069073-e3fe-4c50-aa18-5898937a40c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:26.284185 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d069073-e3fe-4c50-aa18-5898937a40c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:26.284686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d069073-e3fe-4c50-aa18-5898937a40c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:26.286531 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d069073-e3fe-4c50-aa18-5898937a40c8 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:26.309831 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 477/971] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:26.311578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec0ba46-fe95-4410-8d0c-f83d93a8af0c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:26.410594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:26.472667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec0ba46-fe95-4410-8d0c-f83d93a8af0c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:26.530974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdc627-df05-499b-9986-10fda62b7338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:26.531660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdc627-df05-499b-9986-10fda62b7338 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:26.532074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdc627-df05-499b-9986-10fda62b7338 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:26.532551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdc627-df05-499b-9986-10fda62b7338 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:26.545580 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdc627-df05-499b-9986-10fda62b7338 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:26.547100 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdc627-df05-499b-9986-10fda62b7338 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-6fcdc627-df05-499b-9986-10fda62b7338 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:26.560114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:26.561344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:26.562311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:26.562311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:26.562311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:26.562311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:26.562311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:26.562311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:26.562311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:26.562311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:26.562311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:29:26.562311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:26.566487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 495/972] 10.222.0.70 () {56 vars in 1117 bytes} [Wed Aug 24 16:29:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:26.567529 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d069073-e3fe-4c50-aa18-5898937a40c8 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:26.617925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:26.705964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d069073-e3fe-4c50-aa18-5898937a40c8 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:26.753807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b16409-3ac9-45f8-94b2-123cfa180061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:26.754444 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b16409-3ac9-45f8-94b2-123cfa180061 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:26.754728 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b16409-3ac9-45f8-94b2-123cfa180061 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:26.755005 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b16409-3ac9-45f8-94b2-123cfa180061 None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:26.755779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b16409-3ac9-45f8-94b2-123cfa180061 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:26.756087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b16409-3ac9-45f8-94b2-123cfa180061 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:26.757643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b16409-3ac9-45f8-94b2-123cfa180061 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:26.781109 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 478/973] 10.222.0.70 () {56 vars in 1149 bytes} [Wed Aug 24 16:29:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:26.789096 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdc627-df05-499b-9986-10fda62b7338 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:26.915088 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3a42b-957a-4360-b3e1-97e00e5d7808 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:26.915548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3a42b-957a-4360-b3e1-97e00e5d7808 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:26.915880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3a42b-957a-4360-b3e1-97e00e5d7808 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:26.918209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 479/974] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:26.924570 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1ee9d-7310-424a-ab52-975321b24e5a None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:26.925038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1ee9d-7310-424a-ab52-975321b24e5a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:26.925504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1ee9d-7310-424a-ab52-975321b24e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:26.952793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-83f1ee9d-7310-424a-ab52-975321b24e5a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:26.954645 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-83f1ee9d-7310-424a-ab52-975321b24e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:26.979023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdc627-df05-499b-9986-10fda62b7338 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:27.005409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:27.018737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382e262-790b-4a78-8b58-d50325bd3efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:27.019371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382e262-790b-4a78-8b58-d50325bd3efa None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:27.019811 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382e262-790b-4a78-8b58-d50325bd3efa None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:27.020093 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382e262-790b-4a78-8b58-d50325bd3efa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:27.042300 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 480/975] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:29:27.120308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382e262-790b-4a78-8b58-d50325bd3efa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:27.122987 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382e262-790b-4a78-8b58-d50325bd3efa None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:27.245779 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 496/976] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:26 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:29:27.246979 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1ee9d-7310-424a-ab52-975321b24e5a None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:27.288688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:27.321815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382e262-790b-4a78-8b58-d50325bd3efa None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:27.352821 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:27.381818 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1ee9d-7310-424a-ab52-975321b24e5a None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:27.431534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc100a-cf74-463e-aad7-92f9cf46fa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:27.432464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc100a-cf74-463e-aad7-92f9cf46fa84 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:27.432878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc100a-cf74-463e-aad7-92f9cf46fa84 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:27.433224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc100a-cf74-463e-aad7-92f9cf46fa84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:27.448699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc100a-cf74-463e-aad7-92f9cf46fa84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:27.450691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc100a-cf74-463e-aad7-92f9cf46fa84 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-13dc100a-cf74-463e-aad7-92f9cf46fa84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:27.470415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:27.471464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:27.472371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:27.472371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:27.472371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:27.472371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 481/977] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:27.479302 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc100a-cf74-463e-aad7-92f9cf46fa84 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:27.480707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382e262-790b-4a78-8b58-d50325bd3efa None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:27.528984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968df41-1484-4b20-95e5-1937aeb376c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:27.529631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968df41-1484-4b20-95e5-1937aeb376c5 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:27.529908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968df41-1484-4b20-95e5-1937aeb376c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:27.530177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968df41-1484-4b20-95e5-1937aeb376c5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:27.530764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968df41-1484-4b20-95e5-1937aeb376c5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:27.532128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968df41-1484-4b20-95e5-1937aeb376c5 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-c968df41-1484-4b20-95e5-1937aeb376c5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:27.543189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:27.544167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:27.550136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:27.550136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:27.550136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:27.550136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:27.550136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:27.550136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:27.550136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:27.550136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:27.550136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:27.550136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:27.550136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:27.550136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:27.550136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:29:27.550136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:27.550136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 497/978] 10.222.0.70 () {56 vars in 1126 bytes} [Wed Aug 24 16:29:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:27.551527 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968df41-1484-4b20-95e5-1937aeb376c5 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:27.617273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:27.641872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc100a-cf74-463e-aad7-92f9cf46fa84 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:27.668908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:27.684546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c8eec-b800-4d11-9422-b54bda5b9632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:27.685168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c8eec-b800-4d11-9422-b54bda5b9632 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:27.685448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c8eec-b800-4d11-9422-b54bda5b9632 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:27.685725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c8eec-b800-4d11-9422-b54bda5b9632 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:27.686511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c8eec-b800-4d11-9422-b54bda5b9632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:27.686835 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c8eec-b800-4d11-9422-b54bda5b9632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:27.686998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968df41-1484-4b20-95e5-1937aeb376c5 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:27.688214 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c8eec-b800-4d11-9422-b54bda5b9632 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:27.701281 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 482/979] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:27.702591 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce1d2c-3d4d-4905-b1f7-9a08c3a6de50 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:27.702889 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce1d2c-3d4d-4905-b1f7-9a08c3a6de50 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:27.703342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce1d2c-3d4d-4905-b1f7-9a08c3a6de50 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:27.707690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 483/980] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:27.707690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce1d2c-3d4d-4905-b1f7-9a08c3a6de50 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:27.739302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36951b-7405-4eae-844c-13e8036f2196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:27.740030 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36951b-7405-4eae-844c-13e8036f2196 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:27.740343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36951b-7405-4eae-844c-13e8036f2196 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:27.740770 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36951b-7405-4eae-844c-13e8036f2196 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:27.741552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36951b-7405-4eae-844c-13e8036f2196 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:27.742028 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36951b-7405-4eae-844c-13e8036f2196 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:27.743570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36951b-7405-4eae-844c-13e8036f2196 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:27.750720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 498/981] 10.222.0.70 () {56 vars in 1122 bytes} [Wed Aug 24 16:29:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:27.754588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8df031-0879-4096-8d5c-ff5b5bc5bddb None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:27.754588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8df031-0879-4096-8d5c-ff5b5bc5bddb None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:27.754588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8df031-0879-4096-8d5c-ff5b5bc5bddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:27.776133 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-2e8df031-0879-4096-8d5c-ff5b5bc5bddb None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:27.777846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2e8df031-0879-4096-8d5c-ff5b5bc5bddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:27.829756 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce1d2c-3d4d-4905-b1f7-9a08c3a6de50 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:27.866881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4011c0-8d5e-4584-8466-4758350a3b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:27.867598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4011c0-8d5e-4584-8466-4758350a3b53 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:27.867814 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4011c0-8d5e-4584-8466-4758350a3b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:27.868079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4011c0-8d5e-4584-8466-4758350a3b53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:27.871857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 499/982] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:27.873460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8df031-0879-4096-8d5c-ff5b5bc5bddb None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:27.876422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4011c0-8d5e-4584-8466-4758350a3b53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:27.877889 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4011c0-8d5e-4584-8466-4758350a3b53 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:27.890702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-5c4011c0-8d5e-4584-8466-4758350a3b53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:27.890702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:27.890702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:27.890702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:27.890702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:27.890702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:27.890702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:27.890702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:27.890702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:27.890702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:27.890702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:27.890702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:27.890702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:27.890702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:27.890702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:27.891691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:27.892676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:27.892676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:27.892676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:27.892676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:27.892676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:27.892676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:27.892676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:27.892676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:27.892676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 484/983] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:27.897691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4011c0-8d5e-4584-8466-4758350a3b53 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:27.900999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:28.028339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:28.030805 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4011c0-8d5e-4584-8466-4758350a3b53 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:28.066547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8df031-0879-4096-8d5c-ff5b5bc5bddb None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:28.081753 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5db3f-79a0-4958-9e3e-d2f72a994e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:28.082580 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5db3f-79a0-4958-9e3e-d2f72a994e5e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:28.083012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5db3f-79a0-4958-9e3e-d2f72a994e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:28.083339 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5db3f-79a0-4958-9e3e-d2f72a994e5e None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:28.084119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5db3f-79a0-4958-9e3e-d2f72a994e5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:28.084543 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5db3f-79a0-4958-9e3e-d2f72a994e5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:28.086621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5db3f-79a0-4958-9e3e-d2f72a994e5e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:28.111308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 485/984] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:28.112288 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5db3f-79a0-4958-9e3e-d2f72a994e5e None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:28.123748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6644e-7a86-4379-9da2-5de7f5e12ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:28.124274 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6644e-7a86-4379-9da2-5de7f5e12ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:28.124590 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6644e-7a86-4379-9da2-5de7f5e12ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:28.124861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6644e-7a86-4379-9da2-5de7f5e12ba4 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:28.125516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6644e-7a86-4379-9da2-5de7f5e12ba4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:28.125821 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6644e-7a86-4379-9da2-5de7f5e12ba4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:28.127230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6644e-7a86-4379-9da2-5de7f5e12ba4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:28.133722 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 500/985] 10.222.0.70 () {56 vars in 1122 bytes} [Wed Aug 24 16:29:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:28.140075 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6644e-7a86-4379-9da2-5de7f5e12ba4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:28.208680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:28.259796 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5db3f-79a0-4958-9e3e-d2f72a994e5e None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:28.274774 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6644e-7a86-4379-9da2-5de7f5e12ba4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:28.305732 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97f2e2-f99b-4f9f-a12e-a629fc3185e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:28.306347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97f2e2-f99b-4f9f-a12e-a629fc3185e3 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:28.306640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97f2e2-f99b-4f9f-a12e-a629fc3185e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:28.307433 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97f2e2-f99b-4f9f-a12e-a629fc3185e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:28.308179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97f2e2-f99b-4f9f-a12e-a629fc3185e3 None admin] RBAC: Adding query filter params (role.id=23f597bcb1654a0b9575d466bbb86ed6, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=7f3899f2eb554f2fb5a7c43f660b51d5) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:28.308526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97f2e2-f99b-4f9f-a12e-a629fc3185e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:28.309997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97f2e2-f99b-4f9f-a12e-a629fc3185e3 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:28.316613 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 486/986] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:28 2022] GET /identity/v3/role_assignments?user.id=7f3899f2eb554f2fb5a7c43f660b51d5&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=23f597bcb1654a0b9575d466bbb86ed6 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:28.317456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97f2e2-f99b-4f9f-a12e-a629fc3185e3 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:28.331467 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777146d-7f8a-4478-9e90-07004316dfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:28.332434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777146d-7f8a-4478-9e90-07004316dfa5 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:28.332933 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777146d-7f8a-4478-9e90-07004316dfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:28.333343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777146d-7f8a-4478-9e90-07004316dfa5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:28.334562 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777146d-7f8a-4478-9e90-07004316dfa5 None admin] RBAC: Adding query filter params (interface=admin, service_id=3ad17823fce84d3da6fd08524f2958a6, region_id=RegionOne) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:28.334992 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777146d-7f8a-4478-9e90-07004316dfa5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:28.337325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777146d-7f8a-4478-9e90-07004316dfa5 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:28.347488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 501/987] 10.222.0.70 () {56 vars in 1249 bytes} [Wed Aug 24 16:29:28 2022] GET /identity/v3/endpoints?service_id=3ad17823fce84d3da6fd08524f2958a6&interface=admin®ion_id=RegionOne => generated 193 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:28.355428 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777146d-7f8a-4478-9e90-07004316dfa5 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:28.374695 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:28.506782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97f2e2-f99b-4f9f-a12e-a629fc3185e3 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:28.518875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777146d-7f8a-4478-9e90-07004316dfa5 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:28.540943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:28.552495 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c19346-37a0-4509-9c60-50ea1edc0cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:28.553229 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c19346-37a0-4509-9c60-50ea1edc0cd3 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:28.553504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c19346-37a0-4509-9c60-50ea1edc0cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:28.558412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c19346-37a0-4509-9c60-50ea1edc0cd3 None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:28.558412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c19346-37a0-4509-9c60-50ea1edc0cd3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:28.558412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c19346-37a0-4509-9c60-50ea1edc0cd3 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:28.576476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfe2f9-7811-4fc4-bc92-cb8646a2d87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:28.577314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfe2f9-7811-4fc4-bc92-cb8646a2d87f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:28.577651 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfe2f9-7811-4fc4-bc92-cb8646a2d87f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:28.578059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfe2f9-7811-4fc4-bc92-cb8646a2d87f None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:28.578803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfe2f9-7811-4fc4-bc92-cb8646a2d87f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:28.580368 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfe2f9-7811-4fc4-bc92-cb8646a2d87f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:28.587530 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 502/988] 10.222.0.70 () {56 vars in 1087 bytes} [Wed Aug 24 16:29:28 2022] GET /identity/v3/services => generated 1497 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:29:28.654393 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 487/989] 10.222.0.70 () {60 vars in 1141 bytes} [Wed Aug 24 16:29:28 2022] POST /identity/v3/services => generated 242 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:30.366886 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e75b4f-df0c-4cc1-aad1-af0cc26f3d62 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:30.367424 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e75b4f-df0c-4cc1-aad1-af0cc26f3d62 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:30.367868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e75b4f-df0c-4cc1-aad1-af0cc26f3d62 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:30.370498 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 503/990] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:30.376613 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b6307-bfe1-42e9-878c-f765b90d9d08 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:30.377409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b6307-bfe1-42e9-878c-f765b90d9d08 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:30.377409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b6307-bfe1-42e9-878c-f765b90d9d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:30.397418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-362b6307-bfe1-42e9-878c-f765b90d9d08 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:30.398652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-362b6307-bfe1-42e9-878c-f765b90d9d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:30.490418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 488/991] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:29:30.659902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baaf3e-1e5f-4745-8410-e64a2cb35a87 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:30.660228 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baaf3e-1e5f-4745-8410-e64a2cb35a87 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:30.660433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baaf3e-1e5f-4745-8410-e64a2cb35a87 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:30.661941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 504/992] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:30.667766 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e70ca-057e-4803-8f34-8c62f7a25272 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:30.668054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e70ca-057e-4803-8f34-8c62f7a25272 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:30.668330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e70ca-057e-4803-8f34-8c62f7a25272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:30.691772 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-396e70ca-057e-4803-8f34-8c62f7a25272 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:30.693038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-396e70ca-057e-4803-8f34-8c62f7a25272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:30.779318 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 489/993] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:30.826754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558af0f8-b7b6-4657-8287-02a3b4c608b3 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:30.827203 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558af0f8-b7b6-4657-8287-02a3b4c608b3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:30.827668 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558af0f8-b7b6-4657-8287-02a3b4c608b3 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:30.830277 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 505/994] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:30.838741 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0d6bd-38a4-4e22-8fd9-fc5991232e17 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:30.839213 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0d6bd-38a4-4e22-8fd9-fc5991232e17 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:30.839585 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0d6bd-38a4-4e22-8fd9-fc5991232e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:30.869997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-1de0d6bd-38a4-4e22-8fd9-fc5991232e17 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:30.871948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1de0d6bd-38a4-4e22-8fd9-fc5991232e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:30.925799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:30.940067 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558af0f8-b7b6-4657-8287-02a3b4c608b3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:30.971271 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 490/995] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:31.045628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:31.061414 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0d6bd-38a4-4e22-8fd9-fc5991232e17 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:31.096379 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558af0f8-b7b6-4657-8287-02a3b4c608b3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:31.153813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb4666-3129-4227-be3c-bd64aa5d7672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:31.154616 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb4666-3129-4227-be3c-bd64aa5d7672 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:31.154985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb4666-3129-4227-be3c-bd64aa5d7672 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:31.155387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb4666-3129-4227-be3c-bd64aa5d7672 None admin] PATH_INFO: `/v3/services/image` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:31.156203 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb4666-3129-4227-be3c-bd64aa5d7672 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:31.158206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb4666-3129-4227-be3c-bd64aa5d7672 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-20cb4666-3129-4227-be3c-bd64aa5d7672 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:31.171097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:31.173132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:31.174812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:31.174812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:31.174812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:31.174812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:31.174812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:31.174812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:31.174812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:31.174812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:31.174812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:31.174812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:31.174812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:31.174812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:31.174812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:29:31.174812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:31.174812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 506/996] 10.222.0.70 () {56 vars in 1105 bytes} [Wed Aug 24 16:29:30 2022] GET /identity/v3/services/image => generated 86 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:31.178778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb4666-3129-4227-be3c-bd64aa5d7672 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:31.217437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0d6bd-38a4-4e22-8fd9-fc5991232e17 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:31.268093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ce0dd-33ae-48a8-bb0d-942ca9b4949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:31.268784 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ce0dd-33ae-48a8-bb0d-942ca9b4949e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:31.269065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ce0dd-33ae-48a8-bb0d-942ca9b4949e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:31.269349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ce0dd-33ae-48a8-bb0d-942ca9b4949e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:31.269953 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ce0dd-33ae-48a8-bb0d-942ca9b4949e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:31.271446 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ce0dd-33ae-48a8-bb0d-942ca9b4949e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-495ce0dd-33ae-48a8-bb0d-942ca9b4949e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:31.281330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:31.282439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:31.283505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:31.283505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:31.283505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:31.283505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:31.283505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:31.283505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:31.283505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:31.283505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:31.283505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:31.283505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:31.283505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:31.283505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:31.283505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:29:31.283505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:31.283505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 491/997] 10.222.0.70 () {56 vars in 1126 bytes} [Wed Aug 24 16:29:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:31.284593 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ce0dd-33ae-48a8-bb0d-942ca9b4949e None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:31.305096 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb4666-3129-4227-be3c-bd64aa5d7672 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:31.326661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:31.354800 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390c1d4-7266-4ad5-909f-9fac86cccaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:31.355843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390c1d4-7266-4ad5-909f-9fac86cccaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:31.356315 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390c1d4-7266-4ad5-909f-9fac86cccaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:31.356760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390c1d4-7266-4ad5-909f-9fac86cccaf1 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:31.357598 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390c1d4-7266-4ad5-909f-9fac86cccaf1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:31.358012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390c1d4-7266-4ad5-909f-9fac86cccaf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:31.359575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390c1d4-7266-4ad5-909f-9fac86cccaf1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:31.362391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:31.366693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 507/998] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:31.367611 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390c1d4-7266-4ad5-909f-9fac86cccaf1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:31.420942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ce0dd-33ae-48a8-bb0d-942ca9b4949e None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:31.471875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672ab01-2fd0-4423-b640-40bb69c02049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:31.472627 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672ab01-2fd0-4423-b640-40bb69c02049 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:31.472975 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672ab01-2fd0-4423-b640-40bb69c02049 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:31.473317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672ab01-2fd0-4423-b640-40bb69c02049 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:31.473994 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672ab01-2fd0-4423-b640-40bb69c02049 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:31.475765 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672ab01-2fd0-4423-b640-40bb69c02049 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-f672ab01-2fd0-4423-b640-40bb69c02049 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:31.486270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:31.488317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:31.490297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:31.490297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:31.490297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:31.490297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:31.490297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:31.490297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:31.490297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:31.490297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:31.490297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:31.490297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:31.490297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:31.490297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:31.490297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:29:31.490297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:31.490297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 492/999] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:31.492105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672ab01-2fd0-4423-b640-40bb69c02049 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:31.512317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390c1d4-7266-4ad5-909f-9fac86cccaf1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:31.553825 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92494d-a302-443b-a87f-705b8136782c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:31.554457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92494d-a302-443b-a87f-705b8136782c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:31.554735 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92494d-a302-443b-a87f-705b8136782c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:31.555011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92494d-a302-443b-a87f-705b8136782c None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:31.555671 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92494d-a302-443b-a87f-705b8136782c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:31.555977 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92494d-a302-443b-a87f-705b8136782c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:31.557422 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92494d-a302-443b-a87f-705b8136782c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:31.563415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 508/1000] 10.222.0.70 () {56 vars in 1122 bytes} [Wed Aug 24 16:29:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:31.563949 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92494d-a302-443b-a87f-705b8136782c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:31.623387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:31.627720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672ab01-2fd0-4423-b640-40bb69c02049 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:31.639955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:31.673113 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b632e-3a72-412b-b873-7c9a5fe7cba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:31.673732 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b632e-3a72-412b-b873-7c9a5fe7cba3 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:31.674009 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b632e-3a72-412b-b873-7c9a5fe7cba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:31.674280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b632e-3a72-412b-b873-7c9a5fe7cba3 None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:31.674939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b632e-3a72-412b-b873-7c9a5fe7cba3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:31.675234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b632e-3a72-412b-b873-7c9a5fe7cba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:31.676671 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b632e-3a72-412b-b873-7c9a5fe7cba3 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:31.682074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 493/1001] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:31 2022] GET /identity/v3/services?type=image => generated 122 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:31.682961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b632e-3a72-412b-b873-7c9a5fe7cba3 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:31.692952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92494d-a302-443b-a87f-705b8136782c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:31.738878 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cffba-e970-48b5-a132-a9afdd494624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:31.739745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cffba-e970-48b5-a132-a9afdd494624 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:31.740198 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cffba-e970-48b5-a132-a9afdd494624 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:31.740681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cffba-e970-48b5-a132-a9afdd494624 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:31.741735 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cffba-e970-48b5-a132-a9afdd494624 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:31.742215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cffba-e970-48b5-a132-a9afdd494624 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:31.744596 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cffba-e970-48b5-a132-a9afdd494624 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:31.756460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 509/1002] 10.222.0.70 () {56 vars in 1112 bytes} [Wed Aug 24 16:29:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:31.760617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cffba-e970-48b5-a132-a9afdd494624 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:31.858943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b632e-3a72-412b-b873-7c9a5fe7cba3 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:31.911803 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a42f3e-0d89-436c-9681-d117478d92af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:31.912792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a42f3e-0d89-436c-9681-d117478d92af None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:31.913254 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a42f3e-0d89-436c-9681-d117478d92af None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:31.913698 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a42f3e-0d89-436c-9681-d117478d92af None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:31.914710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a42f3e-0d89-436c-9681-d117478d92af None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:31.915092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a42f3e-0d89-436c-9681-d117478d92af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:31.917363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a42f3e-0d89-436c-9681-d117478d92af None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:31.925770 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cffba-e970-48b5-a132-a9afdd494624 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:31.930943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 494/1003] 10.222.0.70 () {56 vars in 1122 bytes} [Wed Aug 24 16:29:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:31.937017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a42f3e-0d89-436c-9681-d117478d92af None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:31.975742 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:31.987137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:31.997791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1bf7d9-4efc-49de-99bc-e30b10edf6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:31.998308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bf7d9-4efc-49de-99bc-e30b10edf6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:31.998590 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bf7d9-4efc-49de-99bc-e30b10edf6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:31.998855 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bf7d9-4efc-49de-99bc-e30b10edf6f0 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:31.999520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1bf7d9-4efc-49de-99bc-e30b10edf6f0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:31.999826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1bf7d9-4efc-49de-99bc-e30b10edf6f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:32.001324 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1bf7d9-4efc-49de-99bc-e30b10edf6f0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:32.007558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 510/1004] 10.222.0.70 () {56 vars in 1112 bytes} [Wed Aug 24 16:29:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:32.013406 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1bf7d9-4efc-49de-99bc-e30b10edf6f0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:32.086066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a42f3e-0d89-436c-9681-d117478d92af None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:32.136313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818546b-1395-479c-a284-68bd82953412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:32.137413 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818546b-1395-479c-a284-68bd82953412 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:32.137718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818546b-1395-479c-a284-68bd82953412 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:32.138008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818546b-1395-479c-a284-68bd82953412 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:32.138654 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818546b-1395-479c-a284-68bd82953412 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:32.140347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818546b-1395-479c-a284-68bd82953412 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:32.179771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1bf7d9-4efc-49de-99bc-e30b10edf6f0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:32.221695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 495/1005] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:29:31 2022] POST /identity/v3/endpoints => generated 325 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:29:32.243590 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89ca92-b1bd-492f-b42b-1f86cc10d5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:32.244827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89ca92-b1bd-492f-b42b-1f86cc10d5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:32.245112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89ca92-b1bd-492f-b42b-1f86cc10d5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:32.245385 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89ca92-b1bd-492f-b42b-1f86cc10d5a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:32.246134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc89ca92-b1bd-492f-b42b-1f86cc10d5a2 None admin] RBAC: Adding query filter params (interface=public, service_id=0b088e186ec046a282c2c2a0b3042a0d, region_id=RegionOne) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:32.246446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc89ca92-b1bd-492f-b42b-1f86cc10d5a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:32.247964 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc89ca92-b1bd-492f-b42b-1f86cc10d5a2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:32.254969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 511/1006] 10.222.0.70 () {56 vars in 1251 bytes} [Wed Aug 24 16:29:32 2022] GET /identity/v3/endpoints?service_id=0b088e186ec046a282c2c2a0b3042a0d&interface=public®ion_id=RegionOne => generated 194 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:32.261716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818546b-1395-479c-a284-68bd82953412 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:32.425238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:32.450717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818546b-1395-479c-a284-68bd82953412 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:32.503977 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c449361-1b32-490c-8484-7fa82fe2f338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:32.504586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c449361-1b32-490c-8484-7fa82fe2f338 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:32.505182 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c449361-1b32-490c-8484-7fa82fe2f338 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:32.505473 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c449361-1b32-490c-8484-7fa82fe2f338 None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:32.506478 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c449361-1b32-490c-8484-7fa82fe2f338 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:32.508662 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c449361-1b32-490c-8484-7fa82fe2f338 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:32.516361 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 496/1007] 10.222.0.70 () {56 vars in 1087 bytes} [Wed Aug 24 16:29:32 2022] GET /identity/v3/services => generated 1727 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:29:33.764275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a96e7-e269-48b4-8da3-107871f7737a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:33.765254 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a96e7-e269-48b4-8da3-107871f7737a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:33.765254 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a96e7-e269-48b4-8da3-107871f7737a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:33.768460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 512/1008] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:33.776798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faea0a-bb5f-4fbb-a342-3c80f2260924 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:33.777325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faea0a-bb5f-4fbb-a342-3c80f2260924 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:33.777789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faea0a-bb5f-4fbb-a342-3c80f2260924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:33.802685 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-77faea0a-bb5f-4fbb-a342-3c80f2260924 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:33.803798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-77faea0a-bb5f-4fbb-a342-3c80f2260924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:33.901875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 497/1009] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:33 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:34.247409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a96e7-e269-48b4-8da3-107871f7737a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:34.333691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:34.389574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8de46-3df9-48ad-ac0e-dfb84012cc73 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:34.389871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8de46-3df9-48ad-ac0e-dfb84012cc73 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:34.390412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8de46-3df9-48ad-ac0e-dfb84012cc73 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:34.391789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 498/1010] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:29:34.393685 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a96e7-e269-48b4-8da3-107871f7737a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:34.402856 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c544f5-0ee2-492d-803a-bc19df889325 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:34.403307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c544f5-0ee2-492d-803a-bc19df889325 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:34.403755 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c544f5-0ee2-492d-803a-bc19df889325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:34.426568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-f0c544f5-0ee2-492d-803a-bc19df889325 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:34.427803 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-f0c544f5-0ee2-492d-803a-bc19df889325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:34.438775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11b4b1-3bae-4413-99f0-78c7a3e9b5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:34.439731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11b4b1-3bae-4413-99f0-78c7a3e9b5a4 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:34.440025 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11b4b1-3bae-4413-99f0-78c7a3e9b5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:34.440471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11b4b1-3bae-4413-99f0-78c7a3e9b5a4 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:34.441295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11b4b1-3bae-4413-99f0-78c7a3e9b5a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:34.442867 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11b4b1-3bae-4413-99f0-78c7a3e9b5a4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:34.513587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 513/1011] 10.222.0.70 () {60 vars in 1141 bytes} [Wed Aug 24 16:29:34 2022] POST /identity/v3/services => generated 242 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:34.530385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 499/1012] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:34 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:34.667804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22d2c9-5dff-488c-a424-776104ad2553 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:34.668113 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22d2c9-5dff-488c-a424-776104ad2553 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:34.668383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22d2c9-5dff-488c-a424-776104ad2553 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:34.669995 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 514/1013] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:34.676594 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694af564-8e2e-4d96-a93a-7ca400e4c65a None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:34.677063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694af564-8e2e-4d96-a93a-7ca400e4c65a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:34.677353 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694af564-8e2e-4d96-a93a-7ca400e4c65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:34.698069 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-694af564-8e2e-4d96-a93a-7ca400e4c65a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:34.699311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-694af564-8e2e-4d96-a93a-7ca400e4c65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:34.701898 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:34.791954 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 500/1014] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:34 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:34.810790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22d2c9-5dff-488c-a424-776104ad2553 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:34.933668 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22d2c9-5dff-488c-a424-776104ad2553 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:34.947870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:34.971511 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f20261-a4c2-4da1-8406-7f26ea65ae1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:34.972146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f20261-a4c2-4da1-8406-7f26ea65ae1f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:34.972445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f20261-a4c2-4da1-8406-7f26ea65ae1f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:34.972718 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f20261-a4c2-4da1-8406-7f26ea65ae1f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:34.973966 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f20261-a4c2-4da1-8406-7f26ea65ae1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:34.975473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f20261-a4c2-4da1-8406-7f26ea65ae1f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:34.977311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 515/1015] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:34.983964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694af564-8e2e-4d96-a93a-7ca400e4c65a None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:35.102994 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694af564-8e2e-4d96-a93a-7ca400e4c65a None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:35.142010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e537edb-3163-4b42-b0d4-d6aaac32eb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:35.142635 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e537edb-3163-4b42-b0d4-d6aaac32eb42 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:35.142919 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e537edb-3163-4b42-b0d4-d6aaac32eb42 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:35.143305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e537edb-3163-4b42-b0d4-d6aaac32eb42 None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:35.143991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e537edb-3163-4b42-b0d4-d6aaac32eb42 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:35.145462 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e537edb-3163-4b42-b0d4-d6aaac32eb42 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:35.156424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:35.264223 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f20261-a4c2-4da1-8406-7f26ea65ae1f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:35.280137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 501/1016] 10.222.0.70 () {60 vars in 1141 bytes} [Wed Aug 24 16:29:34 2022] POST /identity/v3/projects => generated 311 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:35.417542 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f20261-a4c2-4da1-8406-7f26ea65ae1f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:35.463589 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d723f82-4938-48c6-8950-4607dfd2d1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:35.464310 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d723f82-4938-48c6-8950-4607dfd2d1df None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:35.464614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d723f82-4938-48c6-8950-4607dfd2d1df None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:35.464894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d723f82-4938-48c6-8950-4607dfd2d1df None admin] PATH_INFO: `/v3/services/compute` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:35.465554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d723f82-4938-48c6-8950-4607dfd2d1df None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:35.466872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d723f82-4938-48c6-8950-4607dfd2d1df None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-5d723f82-4938-48c6-8950-4607dfd2d1df None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:35.477215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:35.479654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:35.481837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:35.481837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:35.481837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:35.481837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:35.481837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:35.481837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:35.481837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:35.481837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:35.481837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:35.481837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:35.481837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:35.481837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:35.481837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:29:35.481837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:35.481837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 516/1017] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:35.488526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e537edb-3163-4b42-b0d4-d6aaac32eb42 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:35.638211 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e537edb-3163-4b42-b0d4-d6aaac32eb42 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:35.681104 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab4d1e-74c1-46f8-92ec-d1d48af09a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:35.681104 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4d1e-74c1-46f8-92ec-d1d48af09a3b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:35.681104 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4d1e-74c1-46f8-92ec-d1d48af09a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:35.681104 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4d1e-74c1-46f8-92ec-d1d48af09a3b None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:35.681720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab4d1e-74c1-46f8-92ec-d1d48af09a3b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:35.681720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab4d1e-74c1-46f8-92ec-d1d48af09a3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:35.682215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab4d1e-74c1-46f8-92ec-d1d48af09a3b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:35.694072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 502/1018] 10.222.0.70 () {56 vars in 1112 bytes} [Wed Aug 24 16:29:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:35.698989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d723f82-4938-48c6-8950-4607dfd2d1df None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:35.768885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:35.855895 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d723f82-4938-48c6-8950-4607dfd2d1df None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:35.901399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16b762-90e8-4dcd-be3b-9c8574ea070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:35.902032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16b762-90e8-4dcd-be3b-9c8574ea070c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:35.902308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16b762-90e8-4dcd-be3b-9c8574ea070c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:35.902579 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16b762-90e8-4dcd-be3b-9c8574ea070c None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:35.903231 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16b762-90e8-4dcd-be3b-9c8574ea070c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:35.903533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16b762-90e8-4dcd-be3b-9c8574ea070c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:35.904958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16b762-90e8-4dcd-be3b-9c8574ea070c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:35.911100 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 517/1019] 10.222.0.70 () {56 vars in 1112 bytes} [Wed Aug 24 16:29:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:35.917060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab4d1e-74c1-46f8-92ec-d1d48af09a3b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:35.981367 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:36.032858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab4d1e-74c1-46f8-92ec-d1d48af09a3b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:36.074120 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a620a-d61b-4f1f-8577-3b77a8476cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:36.074750 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a620a-d61b-4f1f-8577-3b77a8476cc7 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:36.075029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a620a-d61b-4f1f-8577-3b77a8476cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:36.075363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a620a-d61b-4f1f-8577-3b77a8476cc7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:36.075982 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a620a-d61b-4f1f-8577-3b77a8476cc7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:36.077400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a620a-d61b-4f1f-8577-3b77a8476cc7 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:36.148963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 503/1020] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:29:35 2022] POST /identity/v3/endpoints => generated 334 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:36.598813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb1878-4302-4ee2-8570-db6af800479d None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:36.599196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb1878-4302-4ee2-8570-db6af800479d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:36.599366 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb1878-4302-4ee2-8570-db6af800479d None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:36.600968 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 518/1021] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:36.607745 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0ea8e-b43e-48f9-93d2-cb45cb105d01 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:36.608074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0ea8e-b43e-48f9-93d2-cb45cb105d01 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:36.608311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0ea8e-b43e-48f9-93d2-cb45cb105d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:36.626569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-92b0ea8e-b43e-48f9-93d2-cb45cb105d01 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:36.627750 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-92b0ea8e-b43e-48f9-93d2-cb45cb105d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:36.716447 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 504/1022] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:37.031801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb1878-4302-4ee2-8570-db6af800479d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:37.185296 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb1878-4302-4ee2-8570-db6af800479d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:37.197444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:37.227034 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87d23b-0367-40e0-9280-138fcc7962a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:37.227941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87d23b-0367-40e0-9280-138fcc7962a6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:37.229004 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87d23b-0367-40e0-9280-138fcc7962a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:37.229290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87d23b-0367-40e0-9280-138fcc7962a6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:37.229912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87d23b-0367-40e0-9280-138fcc7962a6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:37.231329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87d23b-0367-40e0-9280-138fcc7962a6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-7d87d23b-0367-40e0-9280-138fcc7962a6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:37.241447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:37.242617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:37.243570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:37.243570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:37.243570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:37.243570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:37.243570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:37.243570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:37.243570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:37.243570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:37.243570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:37.243570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:37.243570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:37.243570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:37.243570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:29:37.243570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:37.248388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 519/1023] 10.222.0.70 () {56 vars in 1105 bytes} [Wed Aug 24 16:29:37 2022] GET /identity/v3/services/image => generated 86 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:37.249884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0ea8e-b43e-48f9-93d2-cb45cb105d01 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:37.306648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:37.402952 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0ea8e-b43e-48f9-93d2-cb45cb105d01 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:37.438178 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97cf2b-921d-40d2-8dbf-0c606828170f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:37.438787 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97cf2b-921d-40d2-8dbf-0c606828170f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:37.439068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97cf2b-921d-40d2-8dbf-0c606828170f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:37.439337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97cf2b-921d-40d2-8dbf-0c606828170f None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:37.440030 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97cf2b-921d-40d2-8dbf-0c606828170f None admin] RBAC: Adding query filter params (name=image) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:37.440334 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97cf2b-921d-40d2-8dbf-0c606828170f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:37.442562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97cf2b-921d-40d2-8dbf-0c606828170f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:37.454752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 505/1024] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:37.456591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87d23b-0367-40e0-9280-138fcc7962a6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:37.529107 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f101259-c26a-43e0-8a76-9f0c7d09e7a2 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:37.529468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f101259-c26a-43e0-8a76-9f0c7d09e7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:37.529671 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f101259-c26a-43e0-8a76-9f0c7d09e7a2 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:37.531005 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 506/1025] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:29:37.536119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ad4c3-fabd-4709-9cb5-c95f57152adb None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:37.536422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ad4c3-fabd-4709-9cb5-c95f57152adb None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:37.536703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ad4c3-fabd-4709-9cb5-c95f57152adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:37.555496 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-3a9ad4c3-fabd-4709-9cb5-c95f57152adb None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:37.556798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3a9ad4c3-fabd-4709-9cb5-c95f57152adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:37.571557 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87d23b-0367-40e0-9280-138fcc7962a6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:37.613164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d601dd-9ed5-4738-8507-799bfda526ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:37.613867 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d601dd-9ed5-4738-8507-799bfda526ce None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:37.614152 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d601dd-9ed5-4738-8507-799bfda526ce None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:37.614422 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d601dd-9ed5-4738-8507-799bfda526ce None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:37.615071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d601dd-9ed5-4738-8507-799bfda526ce None admin] RBAC: Adding query filter params (type=image) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:37.615378 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d601dd-9ed5-4738-8507-799bfda526ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:37.616795 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d601dd-9ed5-4738-8507-799bfda526ce None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:37.622463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 520/1026] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:37.631819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d601dd-9ed5-4738-8507-799bfda526ce None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:37.650848 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 507/1027] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:29:37.670115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:37.737535 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d601dd-9ed5-4738-8507-799bfda526ce None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:37.772988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab9bb4-35c1-4959-8511-e30d80826ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:37.773592 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab9bb4-35c1-4959-8511-e30d80826ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:37.773874 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab9bb4-35c1-4959-8511-e30d80826ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:37.774147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab9bb4-35c1-4959-8511-e30d80826ff6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:37.774851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab9bb4-35c1-4959-8511-e30d80826ff6 None admin] RBAC: Adding query filter params (interface=public, service_id=6e4d792e37124b44868fa8e07241add3, region_id=RegionOne) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:37.775715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab9bb4-35c1-4959-8511-e30d80826ff6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:37.777101 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab9bb4-35c1-4959-8511-e30d80826ff6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:37.786883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 521/1028] 10.222.0.70 () {56 vars in 1251 bytes} [Wed Aug 24 16:29:37 2022] GET /identity/v3/endpoints?service_id=6e4d792e37124b44868fa8e07241add3&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:37.789490 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ad4c3-fabd-4709-9cb5-c95f57152adb None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:37.821054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:37.908042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ad4c3-fabd-4709-9cb5-c95f57152adb None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:37.916711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab9bb4-35c1-4959-8511-e30d80826ff6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:37.971024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60177a-cc4f-41ab-814c-ac20e287a8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:37.971653 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60177a-cc4f-41ab-814c-ac20e287a8ad None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:37.971932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60177a-cc4f-41ab-814c-ac20e287a8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:37.972200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60177a-cc4f-41ab-814c-ac20e287a8ad None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:37.972835 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60177a-cc4f-41ab-814c-ac20e287a8ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:37.974805 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60177a-cc4f-41ab-814c-ac20e287a8ad None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:37.982384 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 508/1029] 10.222.0.70 () {56 vars in 1087 bytes} [Wed Aug 24 16:29:37 2022] GET /identity/v3/services => generated 1957 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:29:38.050820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:38.055112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab9bb4-35c1-4959-8511-e30d80826ff6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:38.093485 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ef013-a1ca-4633-9c6b-1ddf67d91c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:38.094039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ef013-a1ca-4633-9c6b-1ddf67d91c5d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:38.094305 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ef013-a1ca-4633-9c6b-1ddf67d91c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:38.094998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ef013-a1ca-4633-9c6b-1ddf67d91c5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:38.096476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ef013-a1ca-4633-9c6b-1ddf67d91c5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:38.098472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ef013-a1ca-4633-9c6b-1ddf67d91c5d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:38.100296 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 522/1030] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:38.106050 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60177a-cc4f-41ab-814c-ac20e287a8ad None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:38.209569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:38.216414 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60177a-cc4f-41ab-814c-ac20e287a8ad None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:38.256283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9ee6c-bacc-46db-8353-dd4d81ec9dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:38.257093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9ee6c-bacc-46db-8353-dd4d81ec9dbc None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:38.257279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9ee6c-bacc-46db-8353-dd4d81ec9dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:38.257552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9ee6c-bacc-46db-8353-dd4d81ec9dbc None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:38.258232 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9ee6c-bacc-46db-8353-dd4d81ec9dbc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:38.260263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9ee6c-bacc-46db-8353-dd4d81ec9dbc None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:38.318739 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2463b9-c400-4bfe-a83c-2c841cc91619 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:38.319026 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2463b9-c400-4bfe-a83c-2c841cc91619 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:38.319301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2463b9-c400-4bfe-a83c-2c841cc91619 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:38.320733 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 523/1031] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:38.326608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ac701-18c2-4877-b362-4787e50a4bfa None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:38.326889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ac701-18c2-4877-b362-4787e50a4bfa None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:38.327183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ac701-18c2-4877-b362-4787e50a4bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:38.346663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fe2ac701-18c2-4877-b362-4787e50a4bfa None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:38.346663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fe2ac701-18c2-4877-b362-4787e50a4bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:38.374624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 509/1032] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:29:38 2022] POST /identity/v3/users => generated 285 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:29:38.424086 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 524/1033] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:29:38.691390 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9ee6c-bacc-46db-8353-dd4d81ec9dbc None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:38.803517 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9ee6c-bacc-46db-8353-dd4d81ec9dbc None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:38.842428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63eb679-eaca-444f-9037-9098cd268bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:38.843062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63eb679-eaca-444f-9037-9098cd268bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:38.843343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63eb679-eaca-444f-9037-9098cd268bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:38.843613 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63eb679-eaca-444f-9037-9098cd268bd3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:38.855722 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63eb679-eaca-444f-9037-9098cd268bd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-e63eb679-eaca-444f-9037-9098cd268bd3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:38.860567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:38.862771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:38.863995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:38.863995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:38.863995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:38.863995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:38.863995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:38.863995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:38.863995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:38.863995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:38.863995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:38.863995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:38.863995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:29:38.863995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:38.866551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 510/1034] 10.222.0.70 () {56 vars in 1120 bytes} [Wed Aug 24 16:29:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:38.867577 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ac701-18c2-4877-b362-4787e50a4bfa None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:38.976063 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:38.980104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ac701-18c2-4877-b362-4787e50a4bfa None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:39.018225 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24095d3b-6a9a-4a93-b0c4-f2770a408dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:39.019259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24095d3b-6a9a-4a93-b0c4-f2770a408dbb None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:39.019367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24095d3b-6a9a-4a93-b0c4-f2770a408dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:39.019981 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24095d3b-6a9a-4a93-b0c4-f2770a408dbb None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:39.020300 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24095d3b-6a9a-4a93-b0c4-f2770a408dbb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:39.020650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24095d3b-6a9a-4a93-b0c4-f2770a408dbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:39.022034 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24095d3b-6a9a-4a93-b0c4-f2770a408dbb None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:39.036092 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 525/1035] 10.222.0.70 () {56 vars in 1115 bytes} [Wed Aug 24 16:29:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:39.037118 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63eb679-eaca-444f-9037-9098cd268bd3 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:39.059318 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:39.158096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63eb679-eaca-444f-9037-9098cd268bd3 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:39.194317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c47a97-0105-4206-8afd-3083ae95847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:39.194956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c47a97-0105-4206-8afd-3083ae95847c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:39.195230 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c47a97-0105-4206-8afd-3083ae95847c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:39.195504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c47a97-0105-4206-8afd-3083ae95847c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:39.207436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c47a97-0105-4206-8afd-3083ae95847c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:39.211130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c47a97-0105-4206-8afd-3083ae95847c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-e0c47a97-0105-4206-8afd-3083ae95847c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:39.225296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:39.226358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:39.227278 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:39.227278 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:39.227278 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:39.228547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 511/1036] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:39.232828 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24095d3b-6a9a-4a93-b0c4-f2770a408dbb None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:39.357377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24095d3b-6a9a-4a93-b0c4-f2770a408dbb None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:39.392228 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2e19f-6c13-4f7d-b425-fd490baf3a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:39.392884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2e19f-6c13-4f7d-b425-fd490baf3a87 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:39.393161 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2e19f-6c13-4f7d-b425-fd490baf3a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:39.393435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2e19f-6c13-4f7d-b425-fd490baf3a87 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:39.394138 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2e19f-6c13-4f7d-b425-fd490baf3a87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:39.394445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2e19f-6c13-4f7d-b425-fd490baf3a87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:39.395807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2e19f-6c13-4f7d-b425-fd490baf3a87 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:39.414485 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 526/1037] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:39.416370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c47a97-0105-4206-8afd-3083ae95847c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:39.482378 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:39.528923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c47a97-0105-4206-8afd-3083ae95847c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:39.571109 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fdba3-1444-4399-99ed-25be44f2f609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:39.571767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fdba3-1444-4399-99ed-25be44f2f609 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:39.572046 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fdba3-1444-4399-99ed-25be44f2f609 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:39.572313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fdba3-1444-4399-99ed-25be44f2f609 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:39.580008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fdba3-1444-4399-99ed-25be44f2f609 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:39.581461 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fdba3-1444-4399-99ed-25be44f2f609 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-d81fdba3-1444-4399-99ed-25be44f2f609 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:39.592961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:39.593944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:39.594914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:39.594914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:39.594914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:39.594914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:39.594914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:39.594914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:39.594914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:39.594914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:39.594914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:29:39.594914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:39.596087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 512/1038] 10.222.0.70 () {56 vars in 1093 bytes} [Wed Aug 24 16:29:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:39.601698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2e19f-6c13-4f7d-b425-fd490baf3a87 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:39.626704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:39.709425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2e19f-6c13-4f7d-b425-fd490baf3a87 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:39.746438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c5d6d-d44d-4c13-acad-9b9bc3ec00b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:39.746962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c5d6d-d44d-4c13-acad-9b9bc3ec00b4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:39.747237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c5d6d-d44d-4c13-acad-9b9bc3ec00b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:39.747510 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c5d6d-d44d-4c13-acad-9b9bc3ec00b4 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:39.748283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c5d6d-d44d-4c13-acad-9b9bc3ec00b4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:39.748609 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c5d6d-d44d-4c13-acad-9b9bc3ec00b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:39.750130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c5d6d-d44d-4c13-acad-9b9bc3ec00b4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:39.771911 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 527/1039] 10.222.0.70 () {56 vars in 1133 bytes} [Wed Aug 24 16:29:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:39.773271 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fdba3-1444-4399-99ed-25be44f2f609 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:39.881530 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fdba3-1444-4399-99ed-25be44f2f609 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:39.905906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:39.921352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45313e-87c0-4fa4-9aa7-377de27d9e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:39.922205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45313e-87c0-4fa4-9aa7-377de27d9e16 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:39.922649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45313e-87c0-4fa4-9aa7-377de27d9e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:39.923081 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45313e-87c0-4fa4-9aa7-377de27d9e16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:39.959808 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5247fc5-a455-4599-8d33-0bf7447156e6 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:39.960080 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5247fc5-a455-4599-8d33-0bf7447156e6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:39.960354 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5247fc5-a455-4599-8d33-0bf7447156e6 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:39.961741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 528/1040] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:29:39.967755 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7e1d4-6333-4028-a935-0bb139878b32 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:39.968043 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7e1d4-6333-4028-a935-0bb139878b32 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:39.968324 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7e1d4-6333-4028-a935-0bb139878b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:39.986241 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-21f7e1d4-6333-4028-a935-0bb139878b32 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:39.987435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-21f7e1d4-6333-4028-a935-0bb139878b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:40.001340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45313e-87c0-4fa4-9aa7-377de27d9e16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:40.002982 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45313e-87c0-4fa4-9aa7-377de27d9e16 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:40.061524 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 529/1041] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:29:40.116467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 513/1042] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:39 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:29:40.122210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7e1d4-6333-4028-a935-0bb139878b32 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:40.132994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:40.228129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7e1d4-6333-4028-a935-0bb139878b32 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:40.269361 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425515b1-8e19-4b6b-9fc0-f047d57e6fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:40.270054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425515b1-8e19-4b6b-9fc0-f047d57e6fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:40.270299 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425515b1-8e19-4b6b-9fc0-f047d57e6fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:40.270460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425515b1-8e19-4b6b-9fc0-f047d57e6fc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:40.282207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425515b1-8e19-4b6b-9fc0-f047d57e6fc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:40.283660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425515b1-8e19-4b6b-9fc0-f047d57e6fc4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-425515b1-8e19-4b6b-9fc0-f047d57e6fc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:40.298676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:40.299732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:40.305790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:40.305790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:40.305790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:40.305790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 530/1043] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:40.306554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45313e-87c0-4fa4-9aa7-377de27d9e16 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:40.331812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:40.336074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425515b1-8e19-4b6b-9fc0-f047d57e6fc4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:40.404957 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:40.419010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45313e-87c0-4fa4-9aa7-377de27d9e16 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:40.446219 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425515b1-8e19-4b6b-9fc0-f047d57e6fc4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:40.454738 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc87c7-38bc-4667-864a-93fa2b85dd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:40.455347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc87c7-38bc-4667-864a-93fa2b85dd69 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:40.455626 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc87c7-38bc-4667-864a-93fa2b85dd69 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:40.455898 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc87c7-38bc-4667-864a-93fa2b85dd69 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:40.456566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc87c7-38bc-4667-864a-93fa2b85dd69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:40.456868 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc87c7-38bc-4667-864a-93fa2b85dd69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:40.458231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc87c7-38bc-4667-864a-93fa2b85dd69 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:40.469487 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 514/1044] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:40.470777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4902569-bf6b-47d4-8cd4-c422886e9553 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:40.471070 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4902569-bf6b-47d4-8cd4-c422886e9553 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:40.471344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4902569-bf6b-47d4-8cd4-c422886e9553 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:40.472767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 515/1045] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:40.475363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4902569-bf6b-47d4-8cd4-c422886e9553 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:40.481945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cb5bf-55bf-4c82-8bc1-6b9ce0391042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:40.482593 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cb5bf-55bf-4c82-8bc1-6b9ce0391042 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:40.482845 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cb5bf-55bf-4c82-8bc1-6b9ce0391042 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:40.483113 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cb5bf-55bf-4c82-8bc1-6b9ce0391042 None admin] PATH_INFO: `/v3/services/image` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:40.483693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cb5bf-55bf-4c82-8bc1-6b9ce0391042 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:40.485161 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cb5bf-55bf-4c82-8bc1-6b9ce0391042 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-094cb5bf-55bf-4c82-8bc1-6b9ce0391042 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:40.494258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:40.495473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:40.496331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:40.496331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:40.496331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:40.496331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:40.496331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:40.496331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:40.496331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:40.496331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:40.496331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:40.496331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:40.496331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:40.496331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:40.496331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:29:40.496331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:40.496331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 531/1046] 10.222.0.70 () {56 vars in 1105 bytes} [Wed Aug 24 16:29:40 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:40.497157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8916063-69bb-4351-8caa-8866256192fa None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:40.497157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8916063-69bb-4351-8caa-8866256192fa None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:40.497157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8916063-69bb-4351-8caa-8866256192fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:40.515130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b8916063-69bb-4351-8caa-8866256192fa None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:40.516328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b8916063-69bb-4351-8caa-8866256192fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:40.570157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:40.593266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4902569-bf6b-47d4-8cd4-c422886e9553 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:40.594300 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 532/1047] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:40.597578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8916063-69bb-4351-8caa-8866256192fa None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:40.665184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7ba8e-c545-4743-9730-62033faa7246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:40.672939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7ba8e-c545-4743-9730-62033faa7246 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:40.672939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7ba8e-c545-4743-9730-62033faa7246 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:40.672939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7ba8e-c545-4743-9730-62033faa7246 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:40.678888 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7ba8e-c545-4743-9730-62033faa7246 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:40.682261 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7ba8e-c545-4743-9730-62033faa7246 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:40.682990 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:40.697206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-e4c7ba8e-c545-4743-9730-62033faa7246 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:40.697206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:40.697206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:40.697206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:40.697206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:40.697206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:40.697206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:40.697206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:40.697206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:40.697206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:40.697206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:40.697206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:40.697206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:40.697206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:40.697206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:40.701793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:40.702697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:40.702697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:40.702697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:40.702697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:40.702697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:40.702697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:40.702697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:40.702697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:40.702697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 516/1048] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:40.705302 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7ba8e-c545-4743-9730-62033faa7246 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:40.748104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8916063-69bb-4351-8caa-8866256192fa None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:40.794872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c433f6-9a38-4ac1-bc45-ad90a5752746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:40.795425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c433f6-9a38-4ac1-bc45-ad90a5752746 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:40.795706 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c433f6-9a38-4ac1-bc45-ad90a5752746 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:40.795978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c433f6-9a38-4ac1-bc45-ad90a5752746 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:40.797047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c433f6-9a38-4ac1-bc45-ad90a5752746 None admin] RBAC: Adding query filter params (name=image) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:40.797348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c433f6-9a38-4ac1-bc45-ad90a5752746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:40.799039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c433f6-9a38-4ac1-bc45-ad90a5752746 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:40.808750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 533/1049] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:40.810789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c433f6-9a38-4ac1-bc45-ad90a5752746 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:40.831883 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7ba8e-c545-4743-9730-62033faa7246 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:40.871300 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26ab73-8ae9-4c5e-b595-6d70f01b65fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:40.872273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26ab73-8ae9-4c5e-b595-6d70f01b65fb None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:40.872586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26ab73-8ae9-4c5e-b595-6d70f01b65fb None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:40.872914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26ab73-8ae9-4c5e-b595-6d70f01b65fb None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:40.873611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26ab73-8ae9-4c5e-b595-6d70f01b65fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:40.873919 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26ab73-8ae9-4c5e-b595-6d70f01b65fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:40.875346 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26ab73-8ae9-4c5e-b595-6d70f01b65fb None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:40.890284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 517/1050] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:40.891390 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26ab73-8ae9-4c5e-b595-6d70f01b65fb None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:40.894473 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:40.942504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c433f6-9a38-4ac1-bc45-ad90a5752746 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:40.962702 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:40.979855 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510261ab-e80d-44a0-a7cb-03beb3e7ffdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:40.980502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510261ab-e80d-44a0-a7cb-03beb3e7ffdb None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:40.980784 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510261ab-e80d-44a0-a7cb-03beb3e7ffdb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:40.981055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510261ab-e80d-44a0-a7cb-03beb3e7ffdb None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:40.981683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510261ab-e80d-44a0-a7cb-03beb3e7ffdb None admin] RBAC: Adding query filter params (type=image) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:40.981985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510261ab-e80d-44a0-a7cb-03beb3e7ffdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:40.983345 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510261ab-e80d-44a0-a7cb-03beb3e7ffdb None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:40.988094 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 534/1051] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:40.992673 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510261ab-e80d-44a0-a7cb-03beb3e7ffdb None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:41.004535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26ab73-8ae9-4c5e-b595-6d70f01b65fb None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:41.041327 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba7f72-3276-4777-8a6b-b68e4c678637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:41.042096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba7f72-3276-4777-8a6b-b68e4c678637 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:41.042460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba7f72-3276-4777-8a6b-b68e4c678637 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:41.042866 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba7f72-3276-4777-8a6b-b68e4c678637 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:41.054765 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ba7f72-3276-4777-8a6b-b68e4c678637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-e4ba7f72-3276-4777-8a6b-b68e4c678637 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:41.060059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:41.061051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:41.061892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:41.061892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:41.061892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:41.061892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:41.061892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:41.061892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:41.061892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:41.061892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:41.061892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:41.061892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:41.061892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:41.061892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:41.061892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 518/1052] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:29:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:41.062590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba7f72-3276-4777-8a6b-b68e4c678637 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:41.106707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510261ab-e80d-44a0-a7cb-03beb3e7ffdb None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:41.124630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:41.142570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c663a-a000-4ac2-9eca-5c8d0073feab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:41.143175 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c663a-a000-4ac2-9eca-5c8d0073feab None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:41.143449 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c663a-a000-4ac2-9eca-5c8d0073feab None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:41.143719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c663a-a000-4ac2-9eca-5c8d0073feab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:41.144527 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c663a-a000-4ac2-9eca-5c8d0073feab None admin] RBAC: Adding query filter params (role.id=23f597bcb1654a0b9575d466bbb86ed6, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=e333fca7790244f6a4ebceabb55faba4) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:41.144778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c663a-a000-4ac2-9eca-5c8d0073feab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:41.146185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c663a-a000-4ac2-9eca-5c8d0073feab None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:41.152074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 535/1053] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:40 2022] GET /identity/v3/role_assignments?user.id=e333fca7790244f6a4ebceabb55faba4&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=23f597bcb1654a0b9575d466bbb86ed6 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:41.152895 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c663a-a000-4ac2-9eca-5c8d0073feab None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:41.166161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba7f72-3276-4777-8a6b-b68e4c678637 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:41.207085 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:41.210211 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95462c-21ab-4372-b3c4-480471fe6e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:41.210832 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95462c-21ab-4372-b3c4-480471fe6e46 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:41.211108 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95462c-21ab-4372-b3c4-480471fe6e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:41.211377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95462c-21ab-4372-b3c4-480471fe6e46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:41.212107 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95462c-21ab-4372-b3c4-480471fe6e46 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:41.214066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95462c-21ab-4372-b3c4-480471fe6e46 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:41.267456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c663a-a000-4ac2-9eca-5c8d0073feab None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:41.305773 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4569cc7-d859-4130-91e1-d871a283832b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:41.306424 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4569cc7-d859-4130-91e1-d871a283832b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:41.306617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4569cc7-d859-4130-91e1-d871a283832b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:41.306892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4569cc7-d859-4130-91e1-d871a283832b None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:41.307569 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4569cc7-d859-4130-91e1-d871a283832b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:41.307874 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4569cc7-d859-4130-91e1-d871a283832b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:41.309294 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4569cc7-d859-4130-91e1-d871a283832b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:41.319245 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 519/1054] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:29:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:41.324903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 536/1055] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:29:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:41.326306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95462c-21ab-4372-b3c4-480471fe6e46 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:41.463245 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95462c-21ab-4372-b3c4-480471fe6e46 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:41.481559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:41.503827 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7698ce-d5d9-437c-8174-7472bd2611a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:41.504498 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7698ce-d5d9-437c-8174-7472bd2611a4 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:41.510306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7698ce-d5d9-437c-8174-7472bd2611a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:41.510306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7698ce-d5d9-437c-8174-7472bd2611a4 None admin] PATH_INFO: `/v3/users/ee0741452ad84b9eb8c7e62e14d94244` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:41.520044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7698ce-d5d9-437c-8174-7472bd2611a4 None admin] RBAC: Authorizing `identity:get_user(user_id=ee0741452ad84b9eb8c7e62e14d94244)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:41.521535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7698ce-d5d9-437c-8174-7472bd2611a4 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:41.523272 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 520/1056] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:29:41 2022] GET /identity/v3/users/ee0741452ad84b9eb8c7e62e14d94244 => generated 285 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:41.530339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4569cc7-d859-4130-91e1-d871a283832b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:41.673375 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4569cc7-d859-4130-91e1-d871a283832b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:41.683897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:41.711339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb55701-31de-4057-b41d-ad1cd72ece65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:41.711956 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb55701-31de-4057-b41d-ad1cd72ece65 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:41.712293 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb55701-31de-4057-b41d-ad1cd72ece65 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:41.713112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb55701-31de-4057-b41d-ad1cd72ece65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:41.787584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb55701-31de-4057-b41d-ad1cd72ece65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:41.789192 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb55701-31de-4057-b41d-ad1cd72ece65 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:41.895120 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 537/1057] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 16:29:41.901529 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7698ce-d5d9-437c-8174-7472bd2611a4 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:42.025374 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7698ce-d5d9-437c-8174-7472bd2611a4 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:42.083966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a03db-91c3-4d77-96b8-599a63979e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:42.085107 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a03db-91c3-4d77-96b8-599a63979e82 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:42.085598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a03db-91c3-4d77-96b8-599a63979e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:42.086105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a03db-91c3-4d77-96b8-599a63979e82 None admin] PATH_INFO: `/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:42.088097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a03db-91c3-4d77-96b8-599a63979e82 None admin] RBAC: Authorizing `identity:get_project(project_id=af3c85d1ce554cbea67c61c7fc8f81f2)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:42.090886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a03db-91c3-4d77-96b8-599a63979e82 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:42.095619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 521/1058] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:29:41 2022] GET /identity/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2 => generated 311 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:42.104342 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb55701-31de-4057-b41d-ad1cd72ece65 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:42.115333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:42.223627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb55701-31de-4057-b41d-ad1cd72ece65 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:42.263109 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09395b7-793c-4535-a9db-7a420df3467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:42.263722 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09395b7-793c-4535-a9db-7a420df3467c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:42.264002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09395b7-793c-4535-a9db-7a420df3467c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:42.264272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09395b7-793c-4535-a9db-7a420df3467c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:42.265278 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09395b7-793c-4535-a9db-7a420df3467c None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=af3c85d1ce554cbea67c61c7fc8f81f2, user.id=ee0741452ad84b9eb8c7e62e14d94244) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:42.265585 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09395b7-793c-4535-a9db-7a420df3467c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:42.267005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09395b7-793c-4535-a9db-7a420df3467c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:42.280534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 538/1059] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:42 2022] GET /identity/v3/role_assignments?user.id=ee0741452ad84b9eb8c7e62e14d94244&scope.project.id=af3c85d1ce554cbea67c61c7fc8f81f2&role.id=e7f48343fc1542979ab4459865109e30 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:43.063031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed09005-55ed-4fe9-a549-b9c96688be6a None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:43.063621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed09005-55ed-4fe9-a549-b9c96688be6a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:43.063621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed09005-55ed-4fe9-a549-b9c96688be6a None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:43.065173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 522/1060] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:43.070254 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793e349-3f5e-4d7b-9b54-3fa1f71cd735 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:43.070744 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793e349-3f5e-4d7b-9b54-3fa1f71cd735 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:43.070824 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793e349-3f5e-4d7b-9b54-3fa1f71cd735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:43.090409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e793e349-3f5e-4d7b-9b54-3fa1f71cd735 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:43.091606 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e793e349-3f5e-4d7b-9b54-3fa1f71cd735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:43.161314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 539/1061] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:43.316037 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234a0f-eb08-4a48-a83d-2b38866aa43b None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:43.316217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234a0f-eb08-4a48-a83d-2b38866aa43b None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:43.316906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234a0f-eb08-4a48-a83d-2b38866aa43b None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:43.318147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 523/1062] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:43.324337 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1ae44-cd1a-481a-bab1-cf14261cbd75 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:43.324665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1ae44-cd1a-481a-bab1-cf14261cbd75 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:43.324932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1ae44-cd1a-481a-bab1-cf14261cbd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:43.331956 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:43.347453 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-62c1ae44-cd1a-481a-bab1-cf14261cbd75 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:43.348796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-62c1ae44-cd1a-481a-bab1-cf14261cbd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:43.421915 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 540/1063] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:43.427721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5234a0f-eb08-4a48-a83d-2b38866aa43b None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:43.472951 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:43.533904 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5234a0f-eb08-4a48-a83d-2b38866aa43b None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:43.573122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cd82d-8357-4ec8-9088-1c0452296a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:43.573684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cd82d-8357-4ec8-9088-1c0452296a15 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:43.573966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cd82d-8357-4ec8-9088-1c0452296a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:43.574244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cd82d-8357-4ec8-9088-1c0452296a15 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:43.584858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cd82d-8357-4ec8-9088-1c0452296a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-c17cd82d-8357-4ec8-9088-1c0452296a15 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:43.589551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:43.590481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:43.593338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:43.593338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:43.593338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:43.593338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:43.593338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:43.593338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:43.593338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:43.593338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:43.593338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:43.593338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:43.593338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:29:43.593338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:43.593338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 524/1064] 10.222.0.70 () {56 vars in 1120 bytes} [Wed Aug 24 16:29:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:43.596827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1ae44-cd1a-481a-bab1-cf14261cbd75 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:43.693580 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cd82d-8357-4ec8-9088-1c0452296a15 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:43.704421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1ae44-cd1a-481a-bab1-cf14261cbd75 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:43.737320 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:43.744146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6721a2-10fe-4185-b966-397243287818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:43.744798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6721a2-10fe-4185-b966-397243287818 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:43.745343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6721a2-10fe-4185-b966-397243287818 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:43.745343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6721a2-10fe-4185-b966-397243287818 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:43.746006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6721a2-10fe-4185-b966-397243287818 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:43.746355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6721a2-10fe-4185-b966-397243287818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:43.747684 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6721a2-10fe-4185-b966-397243287818 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:43.761248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 541/1065] 10.222.0.70 () {56 vars in 1115 bytes} [Wed Aug 24 16:29:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:43.762234 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6721a2-10fe-4185-b966-397243287818 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:43.803555 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:43.807618 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cd82d-8357-4ec8-9088-1c0452296a15 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:43.845578 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817fbb4-4f2d-4930-82f8-f9dd9a848ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:43.846226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817fbb4-4f2d-4930-82f8-f9dd9a848ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:43.846515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817fbb4-4f2d-4930-82f8-f9dd9a848ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:43.846793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817fbb4-4f2d-4930-82f8-f9dd9a848ab7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:43.869316 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6721a2-10fe-4185-b966-397243287818 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:43.908516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58d912-c008-4915-bc13-fb5df48ce416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:43.909472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58d912-c008-4915-bc13-fb5df48ce416 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:43.910146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58d912-c008-4915-bc13-fb5df48ce416 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:43.910329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58d912-c008-4915-bc13-fb5df48ce416 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:43.919417 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817fbb4-4f2d-4930-82f8-f9dd9a848ab7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:43.920909 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817fbb4-4f2d-4930-82f8-f9dd9a848ab7 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:43.925377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58d912-c008-4915-bc13-fb5df48ce416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:43.926826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58d912-c008-4915-bc13-fb5df48ce416 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-9c58d912-c008-4915-bc13-fb5df48ce416 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:43.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:43.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:43.945516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:43.945516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:43.945516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:43.945516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 542/1066] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:43.950360 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58d912-c008-4915-bc13-fb5df48ce416 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:44.001180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:44.028816 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:44.036444 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 525/1067] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 16:29:44.039292 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817fbb4-4f2d-4930-82f8-f9dd9a848ab7 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:44.059013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58d912-c008-4915-bc13-fb5df48ce416 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:44.095551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011863a-b03d-49c5-8de2-75e0805cf7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:44.096246 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011863a-b03d-49c5-8de2-75e0805cf7ea None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:44.096457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011863a-b03d-49c5-8de2-75e0805cf7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:44.096738 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011863a-b03d-49c5-8de2-75e0805cf7ea None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:44.097381 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011863a-b03d-49c5-8de2-75e0805cf7ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:44.097683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011863a-b03d-49c5-8de2-75e0805cf7ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:44.099092 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011863a-b03d-49c5-8de2-75e0805cf7ea None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:44.116194 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 543/1068] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:44.117584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011863a-b03d-49c5-8de2-75e0805cf7ea None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:44.145721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817fbb4-4f2d-4930-82f8-f9dd9a848ab7 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:44.185164 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15d5ef-0c1c-4ee0-89b5-01fa0397aca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:44.185797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15d5ef-0c1c-4ee0-89b5-01fa0397aca8 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:44.186075 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15d5ef-0c1c-4ee0-89b5-01fa0397aca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:44.186343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15d5ef-0c1c-4ee0-89b5-01fa0397aca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:44.196537 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db15d5ef-0c1c-4ee0-89b5-01fa0397aca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:44.197990 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db15d5ef-0c1c-4ee0-89b5-01fa0397aca8 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-db15d5ef-0c1c-4ee0-89b5-01fa0397aca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:44.212948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:44.214215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:44.215138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:44.215138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:44.215138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:44.215138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 526/1069] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:44.215758 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c591e-3159-4c24-ae7c-9a9b69cf2e5e None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:44.215984 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c591e-3159-4c24-ae7c-9a9b69cf2e5e None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:44.216258 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c591e-3159-4c24-ae7c-9a9b69cf2e5e None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:44.217651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 527/1070] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:44.220566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c591e-3159-4c24-ae7c-9a9b69cf2e5e None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:44.223814 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011863a-b03d-49c5-8de2-75e0805cf7ea None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:44.232166 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:44.259337 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c424dd4-b01e-4ccd-8814-152ee1cdab3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:44.259967 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c424dd4-b01e-4ccd-8814-152ee1cdab3d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:44.260243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c424dd4-b01e-4ccd-8814-152ee1cdab3d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:44.260547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c424dd4-b01e-4ccd-8814-152ee1cdab3d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:44.268047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c424dd4-b01e-4ccd-8814-152ee1cdab3d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:44.269575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c424dd4-b01e-4ccd-8814-152ee1cdab3d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:44.279514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-1c424dd4-b01e-4ccd-8814-152ee1cdab3d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:44.282936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:44.284027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:44.285046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:44.285046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:44.285046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:44.285046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:44.285046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:44.285046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:44.285046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:44.285046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:44.285046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:29:44.285046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:44.285046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 544/1071] 10.222.0.70 () {56 vars in 1093 bytes} [Wed Aug 24 16:29:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:44.285805 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5487f48-432c-40db-9318-2fe9b6bc8d26 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:44.286064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5487f48-432c-40db-9318-2fe9b6bc8d26 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:44.286338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5487f48-432c-40db-9318-2fe9b6bc8d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:44.304244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e5487f48-432c-40db-9318-2fe9b6bc8d26 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:44.305505 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e5487f48-432c-40db-9318-2fe9b6bc8d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:44.331449 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c591e-3159-4c24-ae7c-9a9b69cf2e5e None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:44.366846 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0ea7e-f2ca-41da-9f34-08facfbe58fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:44.367447 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0ea7e-f2ca-41da-9f34-08facfbe58fa None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:44.367730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0ea7e-f2ca-41da-9f34-08facfbe58fa None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:44.367999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0ea7e-f2ca-41da-9f34-08facfbe58fa None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:44.368660 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0ea7e-f2ca-41da-9f34-08facfbe58fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:44.368958 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0ea7e-f2ca-41da-9f34-08facfbe58fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:44.370308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0ea7e-f2ca-41da-9f34-08facfbe58fa None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:44.372085 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 545/1072] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:44.373078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5487f48-432c-40db-9318-2fe9b6bc8d26 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:44.399006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 528/1073] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:44.476061 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:44.482784 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5487f48-432c-40db-9318-2fe9b6bc8d26 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:44.520741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186628c-fdc7-4cca-9a79-18af7d86c174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:44.521369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186628c-fdc7-4cca-9a79-18af7d86c174 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:44.521648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186628c-fdc7-4cca-9a79-18af7d86c174 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:44.521916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186628c-fdc7-4cca-9a79-18af7d86c174 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:44.522660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186628c-fdc7-4cca-9a79-18af7d86c174 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:44.522956 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186628c-fdc7-4cca-9a79-18af7d86c174 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:44.525170 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186628c-fdc7-4cca-9a79-18af7d86c174 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:44.546001 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 546/1074] 10.222.0.70 () {56 vars in 1133 bytes} [Wed Aug 24 16:29:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:44.551911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0ea7e-f2ca-41da-9f34-08facfbe58fa None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:44.641039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186628c-fdc7-4cca-9a79-18af7d86c174 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:44.670518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0ea7e-f2ca-41da-9f34-08facfbe58fa None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:44.693126 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:44.719957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d8332-f633-4410-982c-babc3bd5a8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:44.720845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d8332-f633-4410-982c-babc3bd5a8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:44.721131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d8332-f633-4410-982c-babc3bd5a8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:44.721402 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d8332-f633-4410-982c-babc3bd5a8b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:44.732036 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d8332-f633-4410-982c-babc3bd5a8b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:44.733563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d8332-f633-4410-982c-babc3bd5a8b9 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-9d7d8332-f633-4410-982c-babc3bd5a8b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:44.748550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:44.749547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:44.750449 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:44.750449 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:44.750449 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:44.750449 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 529/1075] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:44.754728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186628c-fdc7-4cca-9a79-18af7d86c174 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:44.755504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d8332-f633-4410-982c-babc3bd5a8b9 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:44.789070 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1895a23-cf42-4fa3-adb3-4aa79f127550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:44.789697 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1895a23-cf42-4fa3-adb3-4aa79f127550 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:44.789977 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1895a23-cf42-4fa3-adb3-4aa79f127550 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:44.790291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1895a23-cf42-4fa3-adb3-4aa79f127550 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:44.802023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1895a23-cf42-4fa3-adb3-4aa79f127550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f1895a23-cf42-4fa3-adb3-4aa79f127550 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:44.807157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:44.808121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:44.813052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:44.813052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:44.813052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:44.813052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:44.813052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:44.813052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:44.813052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:44.813052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:44.813052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:44.813052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:44.813052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:44.813052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:44.813052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 547/1076] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:29:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:44.814233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1895a23-cf42-4fa3-adb3-4aa79f127550 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:44.830820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:44.861128 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d8332-f633-4410-982c-babc3bd5a8b9 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:44.897983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b2e42-c969-4fef-a25f-e30fd3b60e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:44.898519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b2e42-c969-4fef-a25f-e30fd3b60e11 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:44.898802 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b2e42-c969-4fef-a25f-e30fd3b60e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:44.899109 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b2e42-c969-4fef-a25f-e30fd3b60e11 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:44.899758 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b2e42-c969-4fef-a25f-e30fd3b60e11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:44.900063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b2e42-c969-4fef-a25f-e30fd3b60e11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:44.901486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b2e42-c969-4fef-a25f-e30fd3b60e11 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:44.915240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 530/1077] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:44.921044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b2e42-c969-4fef-a25f-e30fd3b60e11 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:44.922493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1895a23-cf42-4fa3-adb3-4aa79f127550 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:44.953247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:44.968002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961dadc-4943-4084-90b2-aa40fcb724b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:44.969999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961dadc-4943-4084-90b2-aa40fcb724b9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:44.969999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961dadc-4943-4084-90b2-aa40fcb724b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:44.969999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961dadc-4943-4084-90b2-aa40fcb724b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:44.970904 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961dadc-4943-4084-90b2-aa40fcb724b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:44.971399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961dadc-4943-4084-90b2-aa40fcb724b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:44.973784 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961dadc-4943-4084-90b2-aa40fcb724b9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:44.982853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 548/1078] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:29:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:44.988686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961dadc-4943-4084-90b2-aa40fcb724b9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:45.036752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b2e42-c969-4fef-a25f-e30fd3b60e11 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:45.067839 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:45.072250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ccf35-bf91-4dc5-bad2-31d40641f2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:45.072923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ccf35-bf91-4dc5-bad2-31d40641f2fe None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:45.073202 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ccf35-bf91-4dc5-bad2-31d40641f2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:45.073483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ccf35-bf91-4dc5-bad2-31d40641f2fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:45.080498 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ccf35-bf91-4dc5-bad2-31d40641f2fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:45.084424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ccf35-bf91-4dc5-bad2-31d40641f2fe None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:45.091140 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961dadc-4943-4084-90b2-aa40fcb724b9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:45.093365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-160ccf35-bf91-4dc5-bad2-31d40641f2fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:45.093365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:45.093365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:45.093365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:45.093365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:45.093365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:45.093365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:45.093365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:45.093365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:45.093365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:45.093365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:45.093365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:45.093365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:45.093365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:45.093365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:45.094401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:45.095313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:45.095313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:45.095313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:45.095313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:45.095313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:45.095313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:45.095313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:45.095313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:45.095313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 531/1079] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:45.100256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ccf35-bf91-4dc5-bad2-31d40641f2fe None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:45.128843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f623d-f052-468b-b582-fe829b2ac4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:45.129475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f623d-f052-468b-b582-fe829b2ac4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:45.129854 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f623d-f052-468b-b582-fe829b2ac4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:45.130126 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f623d-f052-468b-b582-fe829b2ac4e6 None admin] PATH_INFO: `/v3/users/ee0741452ad84b9eb8c7e62e14d94244` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:45.131274 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f623d-f052-468b-b582-fe829b2ac4e6 None admin] RBAC: Authorizing `identity:get_user(user_id=ee0741452ad84b9eb8c7e62e14d94244)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:45.132947 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f623d-f052-468b-b582-fe829b2ac4e6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:45.134330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 549/1080] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:29:44 2022] GET /identity/v3/users/ee0741452ad84b9eb8c7e62e14d94244 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:45.140264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f623d-f052-468b-b582-fe829b2ac4e6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:45.191872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:45.208638 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ccf35-bf91-4dc5-bad2-31d40641f2fe None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:45.243180 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c503e1-6336-48fe-8c19-0a8953826472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:45.244357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c503e1-6336-48fe-8c19-0a8953826472 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:45.244674 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c503e1-6336-48fe-8c19-0a8953826472 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:45.244953 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c503e1-6336-48fe-8c19-0a8953826472 None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:45.245252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f623d-f052-468b-b582-fe829b2ac4e6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:45.245672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c503e1-6336-48fe-8c19-0a8953826472 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:45.245978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c503e1-6336-48fe-8c19-0a8953826472 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:45.247406 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c503e1-6336-48fe-8c19-0a8953826472 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:45.259672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 532/1081] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:45.265822 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c503e1-6336-48fe-8c19-0a8953826472 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:45.279422 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1695fb-e534-4507-a826-88e5a122230c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:45.280054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1695fb-e534-4507-a826-88e5a122230c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:45.280336 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1695fb-e534-4507-a826-88e5a122230c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:45.280652 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1695fb-e534-4507-a826-88e5a122230c None admin] PATH_INFO: `/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:45.281768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1695fb-e534-4507-a826-88e5a122230c None admin] RBAC: Authorizing `identity:get_project(project_id=af3c85d1ce554cbea67c61c7fc8f81f2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:45.283193 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1695fb-e534-4507-a826-88e5a122230c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:45.284871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 550/1082] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:29:45 2022] GET /identity/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:45.291207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1695fb-e534-4507-a826-88e5a122230c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:45.305728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:45.365315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c503e1-6336-48fe-8c19-0a8953826472 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:45.391485 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1695fb-e534-4507-a826-88e5a122230c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:45.401325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadec77-fd12-49d6-bb91-cf079ff5580b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:45.401989 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadec77-fd12-49d6-bb91-cf079ff5580b None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:45.402272 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadec77-fd12-49d6-bb91-cf079ff5580b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:45.402552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadec77-fd12-49d6-bb91-cf079ff5580b None admin] PATH_INFO: `/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/e333fca7790244f6a4ebceabb55faba4/roles/23f597bcb1654a0b9575d466bbb86ed6` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:45.404801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadec77-fd12-49d6-bb91-cf079ff5580b None admin] RBAC: Authorizing `identity:create_grant(project_id=cc920dce30734ac7a69dc039d8c74ce1, user_id=e333fca7790244f6a4ebceabb55faba4, role_id=23f597bcb1654a0b9575d466bbb86ed6)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:45.406234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadec77-fd12-49d6-bb91-cf079ff5580b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:45.424942 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad04e50-a344-482a-83b7-31db4967cbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:45.425597 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad04e50-a344-482a-83b7-31db4967cbf2 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:45.425891 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad04e50-a344-482a-83b7-31db4967cbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:45.426164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad04e50-a344-482a-83b7-31db4967cbf2 None admin] PATH_INFO: `/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2/users/ee0741452ad84b9eb8c7e62e14d94244/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:45.428070 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad04e50-a344-482a-83b7-31db4967cbf2 None admin] RBAC: Authorizing `identity:create_grant(project_id=af3c85d1ce554cbea67c61c7fc8f81f2, user_id=ee0741452ad84b9eb8c7e62e14d94244, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:45.429520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad04e50-a344-482a-83b7-31db4967cbf2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:45.477769 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 533/1083] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:29:45 2022] PUT /identity/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/e333fca7790244f6a4ebceabb55faba4/roles/23f597bcb1654a0b9575d466bbb86ed6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:29:45.506931 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 551/1084] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:29:45 2022] PUT /identity/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2/users/ee0741452ad84b9eb8c7e62e14d94244/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:29:47.368941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50b6b1-9f10-4049-a68a-1f0c9e26a6dc None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:47.369469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50b6b1-9f10-4049-a68a-1f0c9e26a6dc None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:47.369579 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50b6b1-9f10-4049-a68a-1f0c9e26a6dc None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:47.370841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 534/1085] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:47.376223 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83a911-3032-48c7-bf22-76b75278bb22 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:47.376692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83a911-3032-48c7-bf22-76b75278bb22 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:47.376815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83a911-3032-48c7-bf22-76b75278bb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:47.379153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6ecba-8a06-41b5-b702-159ca23488d4 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:47.379440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6ecba-8a06-41b5-b702-159ca23488d4 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:47.379716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6ecba-8a06-41b5-b702-159ca23488d4 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:47.380986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 535/1086] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:29:47.386432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fef8f-9352-4ee9-af5e-70b24a882c9b None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:47.386720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fef8f-9352-4ee9-af5e-70b24a882c9b None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:47.387000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fef8f-9352-4ee9-af5e-70b24a882c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:47.394301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:47.395978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5b83a911-3032-48c7-bf22-76b75278bb22 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:47.397187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5b83a911-3032-48c7-bf22-76b75278bb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:47.405508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-de8fef8f-9352-4ee9-af5e-70b24a882c9b None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:47.406681 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-de8fef8f-9352-4ee9-af5e-70b24a882c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:47.464941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 552/1087] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:47.474830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 536/1088] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:29:47.737829 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83a911-3032-48c7-bf22-76b75278bb22 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:47.739961 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:47.741302 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fef8f-9352-4ee9-af5e-70b24a882c9b None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:47.845470 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fef8f-9352-4ee9-af5e-70b24a882c9b None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:47.859561 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83a911-3032-48c7-bf22-76b75278bb22 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:47.888581 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7662f4-4453-45b0-94d1-7438002448d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:47.889205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7662f4-4453-45b0-94d1-7438002448d8 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:47.889480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7662f4-4453-45b0-94d1-7438002448d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:47.890001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7662f4-4453-45b0-94d1-7438002448d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:47.891812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:47.896300 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c608486-8b91-4164-985b-57b24a07c0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:47.897000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c608486-8b91-4164-985b-57b24a07c0ec None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:47.897288 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c608486-8b91-4164-985b-57b24a07c0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:47.897566 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c608486-8b91-4164-985b-57b24a07c0ec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:47.902025 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7662f4-4453-45b0-94d1-7438002448d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:47.908041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c608486-8b91-4164-985b-57b24a07c0ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-9d7662f4-4453-45b0-94d1-7438002448d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:47.908602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:47.909652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:47.910507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:47.910507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:47.910507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:47.910507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:47.910507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:47.910507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:47.910507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:47.910507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:47.910507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:47.910507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:47.910507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:47.910507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:47.910507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 537/1089] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:29:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-7c608486-8b91-4164-985b-57b24a07c0ec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:47.912827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:47.913982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:47.914848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:47.914848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:47.914848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:47.914848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:47.914848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:47.914848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:47.914848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:47.914848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:47.914848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:47.914848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:47.914848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:29:47.914848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:47.914848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 553/1090] 10.222.0.70 () {56 vars in 1120 bytes} [Wed Aug 24 16:29:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:47.916058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7662f4-4453-45b0-94d1-7438002448d8 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:47.919863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c608486-8b91-4164-985b-57b24a07c0ec None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:47.981908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:48.019084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7662f4-4453-45b0-94d1-7438002448d8 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:48.022052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c608486-8b91-4164-985b-57b24a07c0ec None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:48.052989 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5f6b6-3c5d-43f6-9d1c-eedf6b7be7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:48.053594 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5f6b6-3c5d-43f6-9d1c-eedf6b7be7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:48.053875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5f6b6-3c5d-43f6-9d1c-eedf6b7be7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:48.054147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5f6b6-3c5d-43f6-9d1c-eedf6b7be7d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:48.054814 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5f6b6-3c5d-43f6-9d1c-eedf6b7be7d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:48.055117 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5f6b6-3c5d-43f6-9d1c-eedf6b7be7d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:48.056872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d2e03-7026-4af0-a292-06c8cfe4572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:48.057029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5f6b6-3c5d-43f6-9d1c-eedf6b7be7d4 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:48.057476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d2e03-7026-4af0-a292-06c8cfe4572a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:48.057748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d2e03-7026-4af0-a292-06c8cfe4572a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:48.058019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d2e03-7026-4af0-a292-06c8cfe4572a None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:48.058674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d2e03-7026-4af0-a292-06c8cfe4572a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:48.058980 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d2e03-7026-4af0-a292-06c8cfe4572a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:48.060323 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d2e03-7026-4af0-a292-06c8cfe4572a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:48.065715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 538/1091] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:29:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:48.073797 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 554/1092] 10.222.0.70 () {56 vars in 1115 bytes} [Wed Aug 24 16:29:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:48.073949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5f6b6-3c5d-43f6-9d1c-eedf6b7be7d4 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:48.074801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d2e03-7026-4af0-a292-06c8cfe4572a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:48.127319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:48.176559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5f6b6-3c5d-43f6-9d1c-eedf6b7be7d4 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:48.189307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d2e03-7026-4af0-a292-06c8cfe4572a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:48.212111 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:48.216290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb75db-f064-4aa2-bc84-d5408799b31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:48.222677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb75db-f064-4aa2-bc84-d5408799b31b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:48.222677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb75db-f064-4aa2-bc84-d5408799b31b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:48.222677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb75db-f064-4aa2-bc84-d5408799b31b None admin] PATH_INFO: `/v3/users/ee0741452ad84b9eb8c7e62e14d94244` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:48.222677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb75db-f064-4aa2-bc84-d5408799b31b None admin] RBAC: Authorizing `identity:get_user(user_id=ee0741452ad84b9eb8c7e62e14d94244)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:48.222677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb75db-f064-4aa2-bc84-d5408799b31b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:48.222677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 539/1093] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:29:48 2022] GET /identity/v3/users/ee0741452ad84b9eb8c7e62e14d94244 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:48.223165 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ce41d-e31c-41ca-b898-bfdf4498156a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:48.223254 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ce41d-e31c-41ca-b898-bfdf4498156a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:48.223496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ce41d-e31c-41ca-b898-bfdf4498156a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:48.223770 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ce41d-e31c-41ca-b898-bfdf4498156a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:48.228156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb75db-f064-4aa2-bc84-d5408799b31b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:48.235269 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ce41d-e31c-41ca-b898-bfdf4498156a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:48.236746 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ce41d-e31c-41ca-b898-bfdf4498156a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-332ce41d-e31c-41ca-b898-bfdf4498156a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:48.251411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:48.252536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:48.253517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:48.253517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:48.253517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:48.253517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 555/1094] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:48.258483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ce41d-e31c-41ca-b898-bfdf4498156a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:48.331312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb75db-f064-4aa2-bc84-d5408799b31b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:48.354939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ce41d-e31c-41ca-b898-bfdf4498156a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:48.355303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:48.365944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad5634-60e2-4689-b5ce-b7331447bfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:48.366655 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad5634-60e2-4689-b5ce-b7331447bfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:48.366939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad5634-60e2-4689-b5ce-b7331447bfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:48.367215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad5634-60e2-4689-b5ce-b7331447bfc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:48.388560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c024d4-07ea-454b-b52c-11d3f81b5759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:48.389165 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c024d4-07ea-454b-b52c-11d3f81b5759 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:48.389437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c024d4-07ea-454b-b52c-11d3f81b5759 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:48.389970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c024d4-07ea-454b-b52c-11d3f81b5759 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:48.390599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c024d4-07ea-454b-b52c-11d3f81b5759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:48.390898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c024d4-07ea-454b-b52c-11d3f81b5759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:48.392272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c024d4-07ea-454b-b52c-11d3f81b5759 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:48.403663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 556/1095] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:48.409457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c024d4-07ea-454b-b52c-11d3f81b5759 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:48.439063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad5634-60e2-4689-b5ce-b7331447bfc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:48.440123 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad5634-60e2-4689-b5ce-b7331447bfc6 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:48.445012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:48.510639 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c024d4-07ea-454b-b52c-11d3f81b5759 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:48.542515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 540/1096] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 16:29:48.548712 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ce68e-6552-4f33-a861-01f8b9f4f683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:48.549049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad5634-60e2-4689-b5ce-b7331447bfc6 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:48.549346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ce68e-6552-4f33-a861-01f8b9f4f683 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:48.549629 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ce68e-6552-4f33-a861-01f8b9f4f683 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:48.549903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ce68e-6552-4f33-a861-01f8b9f4f683 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:48.558219 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ce68e-6552-4f33-a861-01f8b9f4f683 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:48.559655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ce68e-6552-4f33-a861-01f8b9f4f683 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-3c7ce68e-6552-4f33-a861-01f8b9f4f683 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:48.571253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:48.572285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:48.578494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:48.578494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:48.578494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:48.578494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:48.578494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:48.578494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:48.578494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:48.578494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:48.578494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:29:48.578494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:29:48.578494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 557/1097] 10.222.0.70 () {56 vars in 1093 bytes} [Wed Aug 24 16:29:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:48.579138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:48.579216 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ce68e-6552-4f33-a861-01f8b9f4f683 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:48.651670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad5634-60e2-4689-b5ce-b7331447bfc6 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:48.673123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:48.678913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ce68e-6552-4f33-a861-01f8b9f4f683 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:48.687805 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af33b1-6c2b-4337-ae82-174a61a66d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:48.688539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af33b1-6c2b-4337-ae82-174a61a66d85 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:48.688822 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af33b1-6c2b-4337-ae82-174a61a66d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:48.689094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af33b1-6c2b-4337-ae82-174a61a66d85 None admin] PATH_INFO: `/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:48.690205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af33b1-6c2b-4337-ae82-174a61a66d85 None admin] RBAC: Authorizing `identity:get_project(project_id=af3c85d1ce554cbea67c61c7fc8f81f2)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:48.691629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af33b1-6c2b-4337-ae82-174a61a66d85 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:48.693301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 541/1098] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:29:48 2022] GET /identity/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:48.699270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af33b1-6c2b-4337-ae82-174a61a66d85 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:48.716726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319dd0be-fc83-4393-93d0-623592f49ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:48.717683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dd0be-fc83-4393-93d0-623592f49ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:48.717960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dd0be-fc83-4393-93d0-623592f49ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:48.718237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dd0be-fc83-4393-93d0-623592f49ca8 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:48.718985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319dd0be-fc83-4393-93d0-623592f49ca8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:48.719293 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319dd0be-fc83-4393-93d0-623592f49ca8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:48.720761 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319dd0be-fc83-4393-93d0-623592f49ca8 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:48.738886 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 558/1099] 10.222.0.70 () {56 vars in 1133 bytes} [Wed Aug 24 16:29:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:48.744656 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319dd0be-fc83-4393-93d0-623592f49ca8 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:48.801745 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:48.810500 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af33b1-6c2b-4337-ae82-174a61a66d85 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:48.844672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17af983-9f54-43fe-aa19-8b0ed9bbcf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:48.845277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17af983-9f54-43fe-aa19-8b0ed9bbcf28 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:48.845558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17af983-9f54-43fe-aa19-8b0ed9bbcf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:48.845831 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17af983-9f54-43fe-aa19-8b0ed9bbcf28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:48.845988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319dd0be-fc83-4393-93d0-623592f49ca8 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:48.846548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17af983-9f54-43fe-aa19-8b0ed9bbcf28 None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=af3c85d1ce554cbea67c61c7fc8f81f2, user.id=ee0741452ad84b9eb8c7e62e14d94244) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:48.846853 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17af983-9f54-43fe-aa19-8b0ed9bbcf28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:48.848268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17af983-9f54-43fe-aa19-8b0ed9bbcf28 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:48.854636 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 542/1100] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:48 2022] GET /identity/v3/role_assignments?user.id=ee0741452ad84b9eb8c7e62e14d94244&scope.project.id=af3c85d1ce554cbea67c61c7fc8f81f2&role.id=e7f48343fc1542979ab4459865109e30 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:48.885701 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ba791-4046-45a5-9f63-392fd3bb62f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:48.886331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ba791-4046-45a5-9f63-392fd3bb62f3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:48.886612 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ba791-4046-45a5-9f63-392fd3bb62f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:48.886886 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ba791-4046-45a5-9f63-392fd3bb62f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:48.917034 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:48.965726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ba791-4046-45a5-9f63-392fd3bb62f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:48.967216 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ba791-4046-45a5-9f63-392fd3bb62f3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:49.069530 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 559/1101] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 16:29:49.075516 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17af983-9f54-43fe-aa19-8b0ed9bbcf28 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:49.175338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17af983-9f54-43fe-aa19-8b0ed9bbcf28 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:49.209388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd7265-41fe-4171-880d-9ac45b7214ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:49.210017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd7265-41fe-4171-880d-9ac45b7214ed None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:49.210297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd7265-41fe-4171-880d-9ac45b7214ed None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:49.210563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd7265-41fe-4171-880d-9ac45b7214ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:49.220561 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd7265-41fe-4171-880d-9ac45b7214ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:49.221994 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd7265-41fe-4171-880d-9ac45b7214ed None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-aacd7265-41fe-4171-880d-9ac45b7214ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:49.236857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:49.237828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:49.238717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:49.238717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:49.238717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:49.238717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 543/1102] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:49.244321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ba791-4046-45a5-9f63-392fd3bb62f3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:49.299849 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:49.343557 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ba791-4046-45a5-9f63-392fd3bb62f3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:49.377902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957bd6b-4b53-44ed-bfa4-8c8eab9032f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:49.378500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957bd6b-4b53-44ed-bfa4-8c8eab9032f4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:49.378781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957bd6b-4b53-44ed-bfa4-8c8eab9032f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:49.379056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957bd6b-4b53-44ed-bfa4-8c8eab9032f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:49.379695 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957bd6b-4b53-44ed-bfa4-8c8eab9032f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:49.380000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957bd6b-4b53-44ed-bfa4-8c8eab9032f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:49.381399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957bd6b-4b53-44ed-bfa4-8c8eab9032f4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:49.392816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 560/1103] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:49.398363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd7265-41fe-4171-880d-9ac45b7214ed None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:49.410119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:49.498907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd7265-41fe-4171-880d-9ac45b7214ed None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:49.532733 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62fa50-f46f-460c-9368-9a6ea2be3865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:49.533366 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62fa50-f46f-460c-9368-9a6ea2be3865 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:49.533648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62fa50-f46f-460c-9368-9a6ea2be3865 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:49.533922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62fa50-f46f-460c-9368-9a6ea2be3865 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:49.543255 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62fa50-f46f-460c-9368-9a6ea2be3865 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:49.544747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62fa50-f46f-460c-9368-9a6ea2be3865 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:49.556161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-3b62fa50-f46f-460c-9368-9a6ea2be3865 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:49.556161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:49.556161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:49.556161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:49.556161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:49.556161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:49.556161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:49.556161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:49.556161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:49.556161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:49.556161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:49.556161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:49.556161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:49.556161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:49.556161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:49.562171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:49.563218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:49.563218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:49.563218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:49.563218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:49.563218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:49.563218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:49.563218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:49.563218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:29:49.563218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 544/1104] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:49.564171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957bd6b-4b53-44ed-bfa4-8c8eab9032f4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:49.661124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957bd6b-4b53-44ed-bfa4-8c8eab9032f4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:49.674415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:49.693356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82873849-34cd-40c1-8861-15370679a3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:49.693949 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82873849-34cd-40c1-8861-15370679a3cd None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:49.694224 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82873849-34cd-40c1-8861-15370679a3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:49.694494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82873849-34cd-40c1-8861-15370679a3cd None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:49.695173 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82873849-34cd-40c1-8861-15370679a3cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:49.695472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82873849-34cd-40c1-8861-15370679a3cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:49.697434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82873849-34cd-40c1-8861-15370679a3cd None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:49.708839 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 561/1105] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:49.715607 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62fa50-f46f-460c-9368-9a6ea2be3865 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:49.782639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:49.816064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62fa50-f46f-460c-9368-9a6ea2be3865 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:49.850550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854eff2-0eed-4a00-9aa6-a0b095cc7554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:49.851181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854eff2-0eed-4a00-9aa6-a0b095cc7554 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:49.851463 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854eff2-0eed-4a00-9aa6-a0b095cc7554 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:49.851743 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854eff2-0eed-4a00-9aa6-a0b095cc7554 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:49.852524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854eff2-0eed-4a00-9aa6-a0b095cc7554 None admin] RBAC: Adding query filter params (role.id=23f597bcb1654a0b9575d466bbb86ed6, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=e333fca7790244f6a4ebceabb55faba4) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:49.852863 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854eff2-0eed-4a00-9aa6-a0b095cc7554 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:49.854240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854eff2-0eed-4a00-9aa6-a0b095cc7554 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:49.860509 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 545/1106] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:49 2022] GET /identity/v3/role_assignments?user.id=e333fca7790244f6a4ebceabb55faba4&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=23f597bcb1654a0b9575d466bbb86ed6 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:50.803259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5db45d-9ad0-48cf-bff6-6f5c2df6b0f0 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:50.803825 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5db45d-9ad0-48cf-bff6-6f5c2df6b0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:50.803825 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5db45d-9ad0-48cf-bff6-6f5c2df6b0f0 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:50.805088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 562/1107] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:50.810393 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f22a5-6f28-42fc-93d1-3910aa5f4c6d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:50.810862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f22a5-6f28-42fc-93d1-3910aa5f4c6d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:50.810967 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f22a5-6f28-42fc-93d1-3910aa5f4c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:50.828933 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-aa9f22a5-6f28-42fc-93d1-3910aa5f4c6d None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:50.830165 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-aa9f22a5-6f28-42fc-93d1-3910aa5f4c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:50.896842 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 546/1108] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:51.167736 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5db45d-9ad0-48cf-bff6-6f5c2df6b0f0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:51.265842 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5db45d-9ad0-48cf-bff6-6f5c2df6b0f0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:51.298547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a256e-4a66-4624-93bc-7f1492cf1832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:51.299166 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a256e-4a66-4624-93bc-7f1492cf1832 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:51.299445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a256e-4a66-4624-93bc-7f1492cf1832 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:51.299717 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a256e-4a66-4624-93bc-7f1492cf1832 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:51.300838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a256e-4a66-4624-93bc-7f1492cf1832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:51.302263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a256e-4a66-4624-93bc-7f1492cf1832 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:51.303839 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 563/1109] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:51.310218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f22a5-6f28-42fc-93d1-3910aa5f4c6d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:51.372309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:51.412861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f22a5-6f28-42fc-93d1-3910aa5f4c6d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:51.448808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aaf649-efe4-4925-83e7-2a313bddf7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:51.449319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaf649-efe4-4925-83e7-2a313bddf7d7 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:51.449601 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaf649-efe4-4925-83e7-2a313bddf7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:51.449875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaf649-efe4-4925-83e7-2a313bddf7d7 None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:51.450521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aaf649-efe4-4925-83e7-2a313bddf7d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:51.451941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aaf649-efe4-4925-83e7-2a313bddf7d7 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:51.573604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 547/1110] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:29:51 2022] POST /identity/v3/users => generated 286 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:53.489046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b3572-92d1-4bf2-8bed-f951ab8d3ce1 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:53.489600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b3572-92d1-4bf2-8bed-f951ab8d3ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:53.489600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b3572-92d1-4bf2-8bed-f951ab8d3ce1 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:53.490916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 564/1111] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:53.496932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21652ea2-1205-41e6-85d8-6c28a21151ab None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:53.497378 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21652ea2-1205-41e6-85d8-6c28a21151ab None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:53.497480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21652ea2-1205-41e6-85d8-6c28a21151ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:53.515746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-21652ea2-1205-41e6-85d8-6c28a21151ab None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:53.516971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-21652ea2-1205-41e6-85d8-6c28a21151ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:53.585385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 548/1112] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:53.857022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b3572-92d1-4bf2-8bed-f951ab8d3ce1 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:53.902611 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:53.964012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b3572-92d1-4bf2-8bed-f951ab8d3ce1 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:53.997690 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485ac29-d0cb-45bd-b505-18d555ad6dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:53.998322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485ac29-d0cb-45bd-b505-18d555ad6dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:53.998606 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485ac29-d0cb-45bd-b505-18d555ad6dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:53.998883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485ac29-d0cb-45bd-b505-18d555ad6dc9 None admin] PATH_INFO: `/v3/roles/003cfd128ecf4c238197991ad36f0c9a` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:54.000271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485ac29-d0cb-45bd-b505-18d555ad6dc9 None admin] RBAC: Authorizing `identity:get_role(role_id=003cfd128ecf4c238197991ad36f0c9a)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:54.001732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485ac29-d0cb-45bd-b505-18d555ad6dc9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:54.003055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 565/1113] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:29:53 2022] GET /identity/v3/roles/003cfd128ecf4c238197991ad36f0c9a => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:54.008895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21652ea2-1205-41e6-85d8-6c28a21151ab None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:54.061777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:54.114387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21652ea2-1205-41e6-85d8-6c28a21151ab None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:54.150512 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396f161-aad3-4580-b82b-99cc6433fb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:54.151150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396f161-aad3-4580-b82b-99cc6433fb93 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:54.151433 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396f161-aad3-4580-b82b-99cc6433fb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:54.151709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396f161-aad3-4580-b82b-99cc6433fb93 None admin] PATH_INFO: `/v3/users/353efc9d79bd40ab983c85cf77d8ac7a` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:54.165537 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396f161-aad3-4580-b82b-99cc6433fb93 None admin] RBAC: Authorizing `identity:get_user(user_id=353efc9d79bd40ab983c85cf77d8ac7a)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:54.166994 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396f161-aad3-4580-b82b-99cc6433fb93 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:54.168704 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 549/1114] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:29:54 2022] GET /identity/v3/users/353efc9d79bd40ab983c85cf77d8ac7a => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:54.174598 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485ac29-d0cb-45bd-b505-18d555ad6dc9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:54.273048 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485ac29-d0cb-45bd-b505-18d555ad6dc9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:54.283995 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:54.306239 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8bd96-7d75-4678-9a2c-95417360a22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:54.306834 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8bd96-7d75-4678-9a2c-95417360a22c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:54.307108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8bd96-7d75-4678-9a2c-95417360a22c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:54.307375 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8bd96-7d75-4678-9a2c-95417360a22c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:54.373415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8bd96-7d75-4678-9a2c-95417360a22c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:54.374766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8bd96-7d75-4678-9a2c-95417360a22c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:54.474627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 566/1115] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:29:54.480791 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396f161-aad3-4580-b82b-99cc6433fb93 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:54.580694 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396f161-aad3-4580-b82b-99cc6433fb93 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:54.588214 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:54.614564 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4714e3-5077-4915-95cf-069904e85707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:54.615196 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4714e3-5077-4915-95cf-069904e85707 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:54.615474 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4714e3-5077-4915-95cf-069904e85707 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:54.615754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4714e3-5077-4915-95cf-069904e85707 None admin] PATH_INFO: `/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:54.616892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4714e3-5077-4915-95cf-069904e85707 None admin] RBAC: Authorizing `identity:get_project(project_id=af3c85d1ce554cbea67c61c7fc8f81f2)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:54.618982 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4714e3-5077-4915-95cf-069904e85707 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:54.620644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 550/1116] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:29:54 2022] GET /identity/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:54.626876 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8bd96-7d75-4678-9a2c-95417360a22c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:54.639004 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:54.723262 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8bd96-7d75-4678-9a2c-95417360a22c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:54.758179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ed324-fda1-4641-bfd0-8454f3cdae2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:54.759328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ed324-fda1-4641-bfd0-8454f3cdae2c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:54.759607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ed324-fda1-4641-bfd0-8454f3cdae2c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:54.759983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ed324-fda1-4641-bfd0-8454f3cdae2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:54.760734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ed324-fda1-4641-bfd0-8454f3cdae2c None admin] RBAC: Adding query filter params (role.id=003cfd128ecf4c238197991ad36f0c9a, scope.project.id=af3c85d1ce554cbea67c61c7fc8f81f2, user.id=353efc9d79bd40ab983c85cf77d8ac7a) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:54.761041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ed324-fda1-4641-bfd0-8454f3cdae2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:54.762453 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ed324-fda1-4641-bfd0-8454f3cdae2c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:54.767948 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 567/1117] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:54 2022] GET /identity/v3/role_assignments?user.id=353efc9d79bd40ab983c85cf77d8ac7a&scope.project.id=af3c85d1ce554cbea67c61c7fc8f81f2&role.id=003cfd128ecf4c238197991ad36f0c9a => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:56.630049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00db6ff-f681-4c6c-9b8b-9e5d9ec115b7 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:56.630597 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00db6ff-f681-4c6c-9b8b-9e5d9ec115b7 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:56.630597 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00db6ff-f681-4c6c-9b8b-9e5d9ec115b7 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:56.632087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 551/1118] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:56.637920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476941e-515e-41ab-a0da-b695ad70d76b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:56.638340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476941e-515e-41ab-a0da-b695ad70d76b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:56.638459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476941e-515e-41ab-a0da-b695ad70d76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:56.657193 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3476941e-515e-41ab-a0da-b695ad70d76b None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:56.658374 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3476941e-515e-41ab-a0da-b695ad70d76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:56.722344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 568/1119] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:56.985096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00db6ff-f681-4c6c-9b8b-9e5d9ec115b7 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:57.086385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00db6ff-f681-4c6c-9b8b-9e5d9ec115b7 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:57.120416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad35158-7a0b-481b-bdc5-b1b624a6e6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:57.121055 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad35158-7a0b-481b-bdc5-b1b624a6e6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:57.121336 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad35158-7a0b-481b-bdc5-b1b624a6e6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:57.121611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad35158-7a0b-481b-bdc5-b1b624a6e6b3 None admin] PATH_INFO: `/v3/roles/003cfd128ecf4c238197991ad36f0c9a` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:57.122993 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad35158-7a0b-481b-bdc5-b1b624a6e6b3 None admin] RBAC: Authorizing `identity:get_role(role_id=003cfd128ecf4c238197991ad36f0c9a)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:57.124355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad35158-7a0b-481b-bdc5-b1b624a6e6b3 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:57.125700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 552/1120] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:29:56 2022] GET /identity/v3/roles/003cfd128ecf4c238197991ad36f0c9a => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:57.131609 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476941e-515e-41ab-a0da-b695ad70d76b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:57.134729 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:57.228979 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476941e-515e-41ab-a0da-b695ad70d76b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:57.262626 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4280f-dcb1-4bf9-858f-9e7235d54e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:57.263185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4280f-dcb1-4bf9-858f-9e7235d54e51 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:57.263465 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4280f-dcb1-4bf9-858f-9e7235d54e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:57.263739 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4280f-dcb1-4bf9-858f-9e7235d54e51 None admin] PATH_INFO: `/v3/users/353efc9d79bd40ab983c85cf77d8ac7a` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:57.265141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4280f-dcb1-4bf9-858f-9e7235d54e51 None admin] RBAC: Authorizing `identity:get_user(user_id=353efc9d79bd40ab983c85cf77d8ac7a)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:57.266581 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4280f-dcb1-4bf9-858f-9e7235d54e51 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:57.268149 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 569/1121] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:29:57 2022] GET /identity/v3/users/353efc9d79bd40ab983c85cf77d8ac7a => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:57.274090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad35158-7a0b-481b-bdc5-b1b624a6e6b3 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:57.317538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:57.372490 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad35158-7a0b-481b-bdc5-b1b624a6e6b3 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:57.405054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda556c4-e267-478c-a6b6-884ef069c417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:57.405669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda556c4-e267-478c-a6b6-884ef069c417 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:57.405947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda556c4-e267-478c-a6b6-884ef069c417 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:57.406218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda556c4-e267-478c-a6b6-884ef069c417 None admin] PATH_INFO: `/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:57.407307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda556c4-e267-478c-a6b6-884ef069c417 None admin] RBAC: Authorizing `identity:get_project(project_id=af3c85d1ce554cbea67c61c7fc8f81f2)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:57.408750 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda556c4-e267-478c-a6b6-884ef069c417 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:57.410364 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 553/1122] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:29:57 2022] GET /identity/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:57.416885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4280f-dcb1-4bf9-858f-9e7235d54e51 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:57.485794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:57.514263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4280f-dcb1-4bf9-858f-9e7235d54e51 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:57.548501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfde324-74fa-4ced-b092-c6886f37c89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:57.549247 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfde324-74fa-4ced-b092-c6886f37c89f None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:57.549628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfde324-74fa-4ced-b092-c6886f37c89f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:57.549997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfde324-74fa-4ced-b092-c6886f37c89f None admin] PATH_INFO: `/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2/users/353efc9d79bd40ab983c85cf77d8ac7a/roles/003cfd128ecf4c238197991ad36f0c9a` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:57.552217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfde324-74fa-4ced-b092-c6886f37c89f None admin] RBAC: Authorizing `identity:create_grant(project_id=af3c85d1ce554cbea67c61c7fc8f81f2, user_id=353efc9d79bd40ab983c85cf77d8ac7a, role_id=003cfd128ecf4c238197991ad36f0c9a)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:57.553674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfde324-74fa-4ced-b092-c6886f37c89f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:57.654420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 570/1123] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:29:57 2022] PUT /identity/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2/users/353efc9d79bd40ab983c85cf77d8ac7a/roles/003cfd128ecf4c238197991ad36f0c9a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:29:59.569353 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c476a2-86be-4ef1-8748-a77ef18ff473 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:59.569872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c476a2-86be-4ef1-8748-a77ef18ff473 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:59.569872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c476a2-86be-4ef1-8748-a77ef18ff473 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:59.571731 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 554/1124] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:59.577271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869126d-5af7-42a8-bee9-0428c7a8b95a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:59.577648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869126d-5af7-42a8-bee9-0428c7a8b95a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:59.577829 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869126d-5af7-42a8-bee9-0428c7a8b95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:59.596340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-6869126d-5af7-42a8-bee9-0428c7a8b95a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:59.597693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6869126d-5af7-42a8-bee9-0428c7a8b95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:59.663252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 571/1125] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:29:59.939726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c476a2-86be-4ef1-8748-a77ef18ff473 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:00.039086 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c476a2-86be-4ef1-8748-a77ef18ff473 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:00.050016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:00.072837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd54b4-42b7-4411-adc3-152954c2a211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:00.073463 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd54b4-42b7-4411-adc3-152954c2a211 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:00.073752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd54b4-42b7-4411-adc3-152954c2a211 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:00.074028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd54b4-42b7-4411-adc3-152954c2a211 None admin] PATH_INFO: `/v3/roles/003cfd128ecf4c238197991ad36f0c9a` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:00.075445 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd54b4-42b7-4411-adc3-152954c2a211 None admin] RBAC: Authorizing `identity:get_role(role_id=003cfd128ecf4c238197991ad36f0c9a)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:00.076970 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd54b4-42b7-4411-adc3-152954c2a211 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:00.078822 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 555/1126] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:29:59 2022] GET /identity/v3/roles/003cfd128ecf4c238197991ad36f0c9a => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:00.084354 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869126d-5af7-42a8-bee9-0428c7a8b95a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:00.143945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:00.181642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869126d-5af7-42a8-bee9-0428c7a8b95a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:00.215769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30f98d-2bea-4a5f-87ca-7c2494776d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:00.216322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30f98d-2bea-4a5f-87ca-7c2494776d1e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:00.216643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30f98d-2bea-4a5f-87ca-7c2494776d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:00.216924 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30f98d-2bea-4a5f-87ca-7c2494776d1e None admin] PATH_INFO: `/v3/users/353efc9d79bd40ab983c85cf77d8ac7a` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:00.218049 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30f98d-2bea-4a5f-87ca-7c2494776d1e None admin] RBAC: Authorizing `identity:get_user(user_id=353efc9d79bd40ab983c85cf77d8ac7a)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:00.219478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30f98d-2bea-4a5f-87ca-7c2494776d1e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:00.221107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 572/1127] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:30:00 2022] GET /identity/v3/users/353efc9d79bd40ab983c85cf77d8ac7a => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:00.226959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd54b4-42b7-4411-adc3-152954c2a211 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:00.326577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd54b4-42b7-4411-adc3-152954c2a211 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:00.360016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f39477-1ace-4459-ae27-7ad60834aca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:00.360651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f39477-1ace-4459-ae27-7ad60834aca5 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:00.360938 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f39477-1ace-4459-ae27-7ad60834aca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:00.361209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f39477-1ace-4459-ae27-7ad60834aca5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:00.405778 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:00.426275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f39477-1ace-4459-ae27-7ad60834aca5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:00.427716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f39477-1ace-4459-ae27-7ad60834aca5 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:00.524689 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 556/1128] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:30:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:30:00.530606 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30f98d-2bea-4a5f-87ca-7c2494776d1e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:00.647470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30f98d-2bea-4a5f-87ca-7c2494776d1e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:00.674431 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:00.680499 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cac75-4725-42de-8b8c-acc3921b6442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:00.681119 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cac75-4725-42de-8b8c-acc3921b6442 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:00.681394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cac75-4725-42de-8b8c-acc3921b6442 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:00.681662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cac75-4725-42de-8b8c-acc3921b6442 None admin] PATH_INFO: `/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:00.682751 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cac75-4725-42de-8b8c-acc3921b6442 None admin] RBAC: Authorizing `identity:get_project(project_id=af3c85d1ce554cbea67c61c7fc8f81f2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:00.684162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cac75-4725-42de-8b8c-acc3921b6442 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:00.685795 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 573/1129] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:30:00 2022] GET /identity/v3/projects/af3c85d1ce554cbea67c61c7fc8f81f2 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:00.692033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f39477-1ace-4459-ae27-7ad60834aca5 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:00.771780 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:00.792063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f39477-1ace-4459-ae27-7ad60834aca5 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:00.826550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d95cd5-3e53-4984-9744-ba765b292cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:00.827143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d95cd5-3e53-4984-9744-ba765b292cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:00.827419 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d95cd5-3e53-4984-9744-ba765b292cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:00.827691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d95cd5-3e53-4984-9744-ba765b292cf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:00.828428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d95cd5-3e53-4984-9744-ba765b292cf7 None admin] RBAC: Adding query filter params (role.id=003cfd128ecf4c238197991ad36f0c9a, scope.project.id=af3c85d1ce554cbea67c61c7fc8f81f2, user.id=353efc9d79bd40ab983c85cf77d8ac7a) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:00.828739 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d95cd5-3e53-4984-9744-ba765b292cf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:00.830143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d95cd5-3e53-4984-9744-ba765b292cf7 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:00.835760 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 557/1130] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:30:00 2022] GET /identity/v3/role_assignments?user.id=353efc9d79bd40ab983c85cf77d8ac7a&scope.project.id=af3c85d1ce554cbea67c61c7fc8f81f2&role.id=003cfd128ecf4c238197991ad36f0c9a => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:02.762729 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb2427-1d43-4335-8626-cb9b074fe899 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:02.763320 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb2427-1d43-4335-8626-cb9b074fe899 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:02.763320 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb2427-1d43-4335-8626-cb9b074fe899 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:02.764911 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 574/1131] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:02.770120 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a1436-07a8-4927-a8c7-1359313a0495 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:02.770538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a1436-07a8-4927-a8c7-1359313a0495 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:02.770675 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a1436-07a8-4927-a8c7-1359313a0495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:02.789707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-2b0a1436-07a8-4927-a8c7-1359313a0495 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:02.790943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2b0a1436-07a8-4927-a8c7-1359313a0495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:02.859476 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 558/1132] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:30:03.133298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb2427-1d43-4335-8626-cb9b074fe899 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:03.232142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb2427-1d43-4335-8626-cb9b074fe899 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:03.266246 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993712ab-05bc-42ca-9b03-3cae99ddeca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:03.266866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993712ab-05bc-42ca-9b03-3cae99ddeca0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:03.267149 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993712ab-05bc-42ca-9b03-3cae99ddeca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:03.267417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993712ab-05bc-42ca-9b03-3cae99ddeca0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:03.268551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993712ab-05bc-42ca-9b03-3cae99ddeca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:03.269977 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993712ab-05bc-42ca-9b03-3cae99ddeca0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:03.271544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 575/1133] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:30:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:03.278187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a1436-07a8-4927-a8c7-1359313a0495 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:03.348047 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:03.375591 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a1436-07a8-4927-a8c7-1359313a0495 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:03.409482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedc094-0294-408a-a2ab-69070a6a6312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:03.410017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedc094-0294-408a-a2ab-69070a6a6312 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:03.410299 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedc094-0294-408a-a2ab-69070a6a6312 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:03.410577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedc094-0294-408a-a2ab-69070a6a6312 None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:03.411211 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedc094-0294-408a-a2ab-69070a6a6312 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:03.412667 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedc094-0294-408a-a2ab-69070a6a6312 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:03.476954 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 559/1134] 10.222.0.70 () {60 vars in 1141 bytes} [Wed Aug 24 16:30:03 2022] POST /identity/v3/projects => generated 311 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:30:05.419877 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e44dca-908a-40dc-afc4-c608a802af4b None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:05.420452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e44dca-908a-40dc-afc4-c608a802af4b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:05.420452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e44dca-908a-40dc-afc4-c608a802af4b None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:05.421781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 576/1135] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:05.427188 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610da80e-d0ad-4d92-b58b-aa8b06ca806d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:05.427608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610da80e-d0ad-4d92-b58b-aa8b06ca806d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:05.427748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610da80e-d0ad-4d92-b58b-aa8b06ca806d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:05.446207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-610da80e-d0ad-4d92-b58b-aa8b06ca806d None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:05.447378 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-610da80e-d0ad-4d92-b58b-aa8b06ca806d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:05.512309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 560/1136] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:30:05.778217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e44dca-908a-40dc-afc4-c608a802af4b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:05.842226 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:05.875520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e44dca-908a-40dc-afc4-c608a802af4b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:05.909947 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595d9ec-becc-4950-90a6-03ed9d0e5b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:05.910561 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595d9ec-becc-4950-90a6-03ed9d0e5b1d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:05.910839 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595d9ec-becc-4950-90a6-03ed9d0e5b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:05.911104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595d9ec-becc-4950-90a6-03ed9d0e5b1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:05.912205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595d9ec-becc-4950-90a6-03ed9d0e5b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:05.913653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595d9ec-becc-4950-90a6-03ed9d0e5b1d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:05.915259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 577/1137] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:30:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:05.921311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610da80e-d0ad-4d92-b58b-aa8b06ca806d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:05.975196 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:06.019894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610da80e-d0ad-4d92-b58b-aa8b06ca806d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:06.053262 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8dadd-ac0f-46d8-8bc3-15aa1f9e7437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:06.053806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8dadd-ac0f-46d8-8bc3-15aa1f9e7437 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:06.054089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8dadd-ac0f-46d8-8bc3-15aa1f9e7437 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:06.054364 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8dadd-ac0f-46d8-8bc3-15aa1f9e7437 None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:06.055540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8dadd-ac0f-46d8-8bc3-15aa1f9e7437 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:06.056999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8dadd-ac0f-46d8-8bc3-15aa1f9e7437 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:06.150480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 561/1138] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:30:05 2022] POST /identity/v3/users => generated 286 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:30:08.105572 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584a506-513d-4b47-a151-bba7698968ef None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:08.106177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584a506-513d-4b47-a151-bba7698968ef None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:08.106177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584a506-513d-4b47-a151-bba7698968ef None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:08.107444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 578/1139] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:08.112794 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ca973-d1ed-49e4-830f-4991c164535b None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:08.113228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ca973-d1ed-49e4-830f-4991c164535b None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:08.113334 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ca973-d1ed-49e4-830f-4991c164535b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:08.131513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-221ca973-d1ed-49e4-830f-4991c164535b None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:08.133340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-221ca973-d1ed-49e4-830f-4991c164535b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:08.198050 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 562/1140] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:30:08.469439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584a506-513d-4b47-a151-bba7698968ef None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:08.570896 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584a506-513d-4b47-a151-bba7698968ef None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:08.604533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6a163-4eac-4e4e-8982-7a1a5049e72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:08.605159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6a163-4eac-4e4e-8982-7a1a5049e72c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:08.605432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6a163-4eac-4e4e-8982-7a1a5049e72c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:08.605700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6a163-4eac-4e4e-8982-7a1a5049e72c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:08.607560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:08.616073 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6a163-4eac-4e4e-8982-7a1a5049e72c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-a6e6a163-4eac-4e4e-8982-7a1a5049e72c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:30:08.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:08.623444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:08.624646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:08.624646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:08.624646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:08.624646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:08.624646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:30:08.624646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:30:08.624646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:30:08.624646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:30:08.624646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:30:08.624646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:30:08.624646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:30:08.624646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:30:08.624646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 579/1141] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:30:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:30:08.628638 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ca973-d1ed-49e4-830f-4991c164535b None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:08.669660 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:08.725086 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ca973-d1ed-49e4-830f-4991c164535b None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:08.862971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096572c-e5db-4f3f-b496-ed0096aa9b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:08.863586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096572c-e5db-4f3f-b496-ed0096aa9b44 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:08.863868 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096572c-e5db-4f3f-b496-ed0096aa9b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:08.864138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096572c-e5db-4f3f-b496-ed0096aa9b44 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:08.864823 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096572c-e5db-4f3f-b496-ed0096aa9b44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:08.865126 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096572c-e5db-4f3f-b496-ed0096aa9b44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:08.866483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096572c-e5db-4f3f-b496-ed0096aa9b44 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:08.877147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 563/1142] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:30:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:08.883074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6a163-4eac-4e4e-8982-7a1a5049e72c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:08.993177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6a163-4eac-4e4e-8982-7a1a5049e72c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:09.027326 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e381d-52e3-47e8-8299-0e6ffc412c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:09.027962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e381d-52e3-47e8-8299-0e6ffc412c77 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:09.028245 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e381d-52e3-47e8-8299-0e6ffc412c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:09.028560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e381d-52e3-47e8-8299-0e6ffc412c77 None admin] PATH_INFO: `/v3/users/7ce20cd10ec14aabb00ee52881b4c91b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:09.042766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e381d-52e3-47e8-8299-0e6ffc412c77 None admin] RBAC: Authorizing `identity:get_user(user_id=7ce20cd10ec14aabb00ee52881b4c91b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:09.044223 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e381d-52e3-47e8-8299-0e6ffc412c77 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:09.045942 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 580/1143] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:30:08 2022] GET /identity/v3/users/7ce20cd10ec14aabb00ee52881b4c91b => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:09.052074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096572c-e5db-4f3f-b496-ed0096aa9b44 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:09.157315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096572c-e5db-4f3f-b496-ed0096aa9b44 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:09.158881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:09.194881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d5938-4895-4d3b-9b72-41c5dbafaf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:09.196259 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d5938-4895-4d3b-9b72-41c5dbafaf11 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:09.196745 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d5938-4895-4d3b-9b72-41c5dbafaf11 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:09.197062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d5938-4895-4d3b-9b72-41c5dbafaf11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:09.268572 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d5938-4895-4d3b-9b72-41c5dbafaf11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:09.269922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d5938-4895-4d3b-9b72-41c5dbafaf11 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:09.364528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 564/1144] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:30:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:30:09.370499 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e381d-52e3-47e8-8299-0e6ffc412c77 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:09.422142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:09.467804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e381d-52e3-47e8-8299-0e6ffc412c77 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:09.501166 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35127ec4-8bc4-4fcb-9a7a-fa8aa9c0e956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:09.501788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35127ec4-8bc4-4fcb-9a7a-fa8aa9c0e956 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:09.502067 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35127ec4-8bc4-4fcb-9a7a-fa8aa9c0e956 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:09.502340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35127ec4-8bc4-4fcb-9a7a-fa8aa9c0e956 None admin] PATH_INFO: `/v3/projects/3552781c9daa4deda0b36bb609172702` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:09.503466 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35127ec4-8bc4-4fcb-9a7a-fa8aa9c0e956 None admin] RBAC: Authorizing `identity:get_project(project_id=3552781c9daa4deda0b36bb609172702)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:09.504935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35127ec4-8bc4-4fcb-9a7a-fa8aa9c0e956 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:09.506551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 581/1145] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:30:09 2022] GET /identity/v3/projects/3552781c9daa4deda0b36bb609172702 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:09.513293 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d5938-4895-4d3b-9b72-41c5dbafaf11 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:09.515748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:09.615781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d5938-4895-4d3b-9b72-41c5dbafaf11 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:09.650760 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abccebaf-4bf1-4ff4-bb83-43259002cae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:09.651357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccebaf-4bf1-4ff4-bb83-43259002cae9 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:09.651694 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccebaf-4bf1-4ff4-bb83-43259002cae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:09.651975 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccebaf-4bf1-4ff4-bb83-43259002cae9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:09.652733 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abccebaf-4bf1-4ff4-bb83-43259002cae9 None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=3552781c9daa4deda0b36bb609172702, user.id=7ce20cd10ec14aabb00ee52881b4c91b) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:09.653040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abccebaf-4bf1-4ff4-bb83-43259002cae9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:09.654459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abccebaf-4bf1-4ff4-bb83-43259002cae9 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:09.659768 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 565/1146] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:30:09 2022] GET /identity/v3/role_assignments?user.id=7ce20cd10ec14aabb00ee52881b4c91b&scope.project.id=3552781c9daa4deda0b36bb609172702&role.id=e7f48343fc1542979ab4459865109e30 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:11.554217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9060fa-1842-4989-a070-a0a2773b7126 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:11.554837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9060fa-1842-4989-a070-a0a2773b7126 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:11.555014 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9060fa-1842-4989-a070-a0a2773b7126 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:11.556586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 582/1147] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:11.562303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba835147-02fc-480f-af7a-9e7cfb2246f9 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:11.562778 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba835147-02fc-480f-af7a-9e7cfb2246f9 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:11.563016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba835147-02fc-480f-af7a-9e7cfb2246f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:11.582607 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-ba835147-02fc-480f-af7a-9e7cfb2246f9 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:11.583830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ba835147-02fc-480f-af7a-9e7cfb2246f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:11.646438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:11.654486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 566/1148] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:30:11.931530 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9060fa-1842-4989-a070-a0a2773b7126 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:12.030124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9060fa-1842-4989-a070-a0a2773b7126 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:12.049428 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:12.063245 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2783a-c9b7-4f47-b083-ebaae5884436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:12.063870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2783a-c9b7-4f47-b083-ebaae5884436 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:12.064153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2783a-c9b7-4f47-b083-ebaae5884436 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:12.064450 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2783a-c9b7-4f47-b083-ebaae5884436 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:12.074981 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2783a-c9b7-4f47-b083-ebaae5884436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-8bf2783a-c9b7-4f47-b083-ebaae5884436 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:30:12.079665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:12.080758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:12.081636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:12.081636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:12.081636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:12.081636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:12.081636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:30:12.081636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:30:12.081636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:30:12.081636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:30:12.081636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:30:12.081636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:30:12.081636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:30:12.081636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:30:12.081636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 583/1149] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:30:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:30:12.087070 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba835147-02fc-480f-af7a-9e7cfb2246f9 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:12.185148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba835147-02fc-480f-af7a-9e7cfb2246f9 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:12.217863 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a1202-4ef4-4184-8201-41504d966b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:12.218464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a1202-4ef4-4184-8201-41504d966b85 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:12.218744 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a1202-4ef4-4184-8201-41504d966b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:12.219017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a1202-4ef4-4184-8201-41504d966b85 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:12.219767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a1202-4ef4-4184-8201-41504d966b85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:12.220070 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a1202-4ef4-4184-8201-41504d966b85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:12.221465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a1202-4ef4-4184-8201-41504d966b85 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:12.229885 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 567/1150] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:30:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:12.235765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2783a-c9b7-4f47-b083-ebaae5884436 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:12.332394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2783a-c9b7-4f47-b083-ebaae5884436 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:12.365316 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ec81d-9ce6-4ae3-bb30-e7d99a0146c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:12.365932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ec81d-9ce6-4ae3-bb30-e7d99a0146c4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:12.366209 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ec81d-9ce6-4ae3-bb30-e7d99a0146c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:12.366475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ec81d-9ce6-4ae3-bb30-e7d99a0146c4 None admin] PATH_INFO: `/v3/users/7ce20cd10ec14aabb00ee52881b4c91b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:12.368318 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ec81d-9ce6-4ae3-bb30-e7d99a0146c4 None admin] RBAC: Authorizing `identity:get_user(user_id=7ce20cd10ec14aabb00ee52881b4c91b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:12.369776 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ec81d-9ce6-4ae3-bb30-e7d99a0146c4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:12.371338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 584/1151] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:30:12 2022] GET /identity/v3/users/7ce20cd10ec14aabb00ee52881b4c91b => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:12.377294 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a1202-4ef4-4184-8201-41504d966b85 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:12.440675 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:12.481847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a1202-4ef4-4184-8201-41504d966b85 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:12.516027 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874ee14-68d8-4698-8553-8f6152b5f315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:12.516688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874ee14-68d8-4698-8553-8f6152b5f315 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:12.516969 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874ee14-68d8-4698-8553-8f6152b5f315 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:12.517242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874ee14-68d8-4698-8553-8f6152b5f315 None admin] PATH_INFO: `/v3/projects/3552781c9daa4deda0b36bb609172702` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:12.518381 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6874ee14-68d8-4698-8553-8f6152b5f315 None admin] RBAC: Authorizing `identity:get_project(project_id=3552781c9daa4deda0b36bb609172702)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:12.519820 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6874ee14-68d8-4698-8553-8f6152b5f315 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:12.521505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 568/1152] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:30:12 2022] GET /identity/v3/projects/3552781c9daa4deda0b36bb609172702 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:12.528462 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ec81d-9ce6-4ae3-bb30-e7d99a0146c4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:12.579684 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:12.628500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ec81d-9ce6-4ae3-bb30-e7d99a0146c4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:12.661495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62775362-9047-44b7-9530-a0daeef59f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:12.662134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62775362-9047-44b7-9530-a0daeef59f4f None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:12.662412 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62775362-9047-44b7-9530-a0daeef59f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:12.662682 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62775362-9047-44b7-9530-a0daeef59f4f None admin] PATH_INFO: `/v3/projects/3552781c9daa4deda0b36bb609172702/users/7ce20cd10ec14aabb00ee52881b4c91b/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:12.665221 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62775362-9047-44b7-9530-a0daeef59f4f None admin] RBAC: Authorizing `identity:create_grant(project_id=3552781c9daa4deda0b36bb609172702, user_id=7ce20cd10ec14aabb00ee52881b4c91b, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:12.666627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62775362-9047-44b7-9530-a0daeef59f4f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:12.733662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 585/1153] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:30:12 2022] PUT /identity/v3/projects/3552781c9daa4deda0b36bb609172702/users/7ce20cd10ec14aabb00ee52881b4c91b/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:30:14.619280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723566a7-64a8-4a2d-8770-16cfec39fb23 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:14.619812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723566a7-64a8-4a2d-8770-16cfec39fb23 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:14.619812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723566a7-64a8-4a2d-8770-16cfec39fb23 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:14.621257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 569/1154] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:14.626512 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbeed9c-756d-4aac-8886-cf71ce2b3940 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:14.626888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbeed9c-756d-4aac-8886-cf71ce2b3940 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:14.627065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbeed9c-756d-4aac-8886-cf71ce2b3940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:14.645093 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9cbeed9c-756d-4aac-8886-cf71ce2b3940 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:14.646274 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9cbeed9c-756d-4aac-8886-cf71ce2b3940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:14.711097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 586/1155] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:30:14.978599 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723566a7-64a8-4a2d-8770-16cfec39fb23 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:15.086306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723566a7-64a8-4a2d-8770-16cfec39fb23 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:15.111653 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:15.119920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc603f1d-9c34-4b12-8c68-42686e8de878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:15.120576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc603f1d-9c34-4b12-8c68-42686e8de878 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:15.120859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc603f1d-9c34-4b12-8c68-42686e8de878 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:15.121131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc603f1d-9c34-4b12-8c68-42686e8de878 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:15.131482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc603f1d-9c34-4b12-8c68-42686e8de878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-bc603f1d-9c34-4b12-8c68-42686e8de878 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:30:15.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:15.137828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:15.138679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:15.138679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:15.138679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:15.138679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:15.138679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:30:15.138679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:30:15.138679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:30:15.138679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:30:15.138679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:30:15.138679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:30:15.138679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:30:15.138679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:30:15.138679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 570/1156] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:30:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:30:15.144452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbeed9c-756d-4aac-8886-cf71ce2b3940 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:15.184932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:15.246099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbeed9c-756d-4aac-8886-cf71ce2b3940 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:15.280885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf400b4-e463-4058-95ce-2cdd69708151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:15.281492 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf400b4-e463-4058-95ce-2cdd69708151 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:15.281773 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf400b4-e463-4058-95ce-2cdd69708151 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:15.282045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf400b4-e463-4058-95ce-2cdd69708151 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:15.282705 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf400b4-e463-4058-95ce-2cdd69708151 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:15.283005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf400b4-e463-4058-95ce-2cdd69708151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:15.284369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf400b4-e463-4058-95ce-2cdd69708151 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:15.292702 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 587/1157] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:30:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:15.298955 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc603f1d-9c34-4b12-8c68-42686e8de878 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:15.398759 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc603f1d-9c34-4b12-8c68-42686e8de878 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:15.430992 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7d6f0-0283-4b16-b11f-0ae34b403d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:15.431615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7d6f0-0283-4b16-b11f-0ae34b403d7b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:15.431895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7d6f0-0283-4b16-b11f-0ae34b403d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:15.432168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7d6f0-0283-4b16-b11f-0ae34b403d7b None admin] PATH_INFO: `/v3/users/7ce20cd10ec14aabb00ee52881b4c91b` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:15.433301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7d6f0-0283-4b16-b11f-0ae34b403d7b None admin] RBAC: Authorizing `identity:get_user(user_id=7ce20cd10ec14aabb00ee52881b4c91b)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:15.434724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7d6f0-0283-4b16-b11f-0ae34b403d7b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:15.436312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 571/1158] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:30:15 2022] GET /identity/v3/users/7ce20cd10ec14aabb00ee52881b4c91b => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:15.442261 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf400b4-e463-4058-95ce-2cdd69708151 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:15.538111 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:15.541299 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf400b4-e463-4058-95ce-2cdd69708151 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:15.575312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b32687-b186-4831-b55c-3f1e5f93d140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:15.575925 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b32687-b186-4831-b55c-3f1e5f93d140 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:15.576203 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b32687-b186-4831-b55c-3f1e5f93d140 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:15.576529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b32687-b186-4831-b55c-3f1e5f93d140 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:15.642686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b32687-b186-4831-b55c-3f1e5f93d140 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:15.644115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b32687-b186-4831-b55c-3f1e5f93d140 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:15.735719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:15.739244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 588/1159] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:30:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:30:15.746794 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7d6f0-0283-4b16-b11f-0ae34b403d7b None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:15.801249 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:15.845974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7d6f0-0283-4b16-b11f-0ae34b403d7b None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:15.879927 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00feda5f-602b-48f4-88f6-4ee6e0a797cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:15.880604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00feda5f-602b-48f4-88f6-4ee6e0a797cb None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:15.880889 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00feda5f-602b-48f4-88f6-4ee6e0a797cb None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:15.881164 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00feda5f-602b-48f4-88f6-4ee6e0a797cb None admin] PATH_INFO: `/v3/projects/3552781c9daa4deda0b36bb609172702` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:15.882323 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00feda5f-602b-48f4-88f6-4ee6e0a797cb None admin] RBAC: Authorizing `identity:get_project(project_id=3552781c9daa4deda0b36bb609172702)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:15.883753 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00feda5f-602b-48f4-88f6-4ee6e0a797cb None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:15.885433 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 572/1160] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:30:15 2022] GET /identity/v3/projects/3552781c9daa4deda0b36bb609172702 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:15.891874 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b32687-b186-4831-b55c-3f1e5f93d140 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:16.003436 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b32687-b186-4831-b55c-3f1e5f93d140 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:16.039353 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb786c1-771d-482f-bf5f-d638e21cf450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:16.039953 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb786c1-771d-482f-bf5f-d638e21cf450 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:16.040230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb786c1-771d-482f-bf5f-d638e21cf450 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:16.040533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb786c1-771d-482f-bf5f-d638e21cf450 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:16.041791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb786c1-771d-482f-bf5f-d638e21cf450 None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=3552781c9daa4deda0b36bb609172702, user.id=7ce20cd10ec14aabb00ee52881b4c91b) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:16.041937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb786c1-771d-482f-bf5f-d638e21cf450 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:16.043571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb786c1-771d-482f-bf5f-d638e21cf450 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:16.059058 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 589/1161] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:30:15 2022] GET /identity/v3/role_assignments?user.id=7ce20cd10ec14aabb00ee52881b4c91b&scope.project.id=3552781c9daa4deda0b36bb609172702&role.id=e7f48343fc1542979ab4459865109e30 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:18.017565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857c934-6c38-4a9f-bc39-7edf5a259f92 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:18.018106 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857c934-6c38-4a9f-bc39-7edf5a259f92 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:18.018106 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857c934-6c38-4a9f-bc39-7edf5a259f92 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:18.019440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 573/1162] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:18.025177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bec7a6-fda2-417e-a283-e91f45c04422 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:18.025606 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bec7a6-fda2-417e-a283-e91f45c04422 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:18.025730 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bec7a6-fda2-417e-a283-e91f45c04422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:18.043262 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-78bec7a6-fda2-417e-a283-e91f45c04422 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:18.044465 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-78bec7a6-fda2-417e-a283-e91f45c04422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:18.102678 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:18.110690 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 590/1163] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:30:18.376251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857c934-6c38-4a9f-bc39-7edf5a259f92 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:18.495932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857c934-6c38-4a9f-bc39-7edf5a259f92 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:18.517901 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:18.529584 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111195e7-e068-4a64-893c-1086fb920505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:18.530188 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111195e7-e068-4a64-893c-1086fb920505 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:18.530464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111195e7-e068-4a64-893c-1086fb920505 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:18.539140 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111195e7-e068-4a64-893c-1086fb920505 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:18.597816 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111195e7-e068-4a64-893c-1086fb920505 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:18.599253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111195e7-e068-4a64-893c-1086fb920505 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:18.696373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 574/1164] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:30:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:30:18.702337 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bec7a6-fda2-417e-a283-e91f45c04422 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:18.805244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bec7a6-fda2-417e-a283-e91f45c04422 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:18.840704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0727585-d74a-4b3d-bbb0-9f77115accde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:18.841346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0727585-d74a-4b3d-bbb0-9f77115accde None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:18.841623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0727585-d74a-4b3d-bbb0-9f77115accde None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:18.841897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0727585-d74a-4b3d-bbb0-9f77115accde None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:18.849006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0727585-d74a-4b3d-bbb0-9f77115accde None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:18.850445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0727585-d74a-4b3d-bbb0-9f77115accde None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-b0727585-d74a-4b3d-bbb0-9f77115accde None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:18.862189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:18.863418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:18.864451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:18.864451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:18.864451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:18.864451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:18.864451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:18.864451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:18.864451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:18.864451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:18.864451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:18.864451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:18.864451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 16:30:18.864451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:30:18.864451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 16:30:18.864451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:30:18.864451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 591/1165] 10.222.0.70 () {56 vars in 1117 bytes} [Wed Aug 24 16:30:18 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:30:18.869606 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111195e7-e068-4a64-893c-1086fb920505 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:18.894809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:18.976792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111195e7-e068-4a64-893c-1086fb920505 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:19.011196 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e275592-5531-4d2a-983b-4c1b5fdbda1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:19.011804 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e275592-5531-4d2a-983b-4c1b5fdbda1d None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:19.012085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e275592-5531-4d2a-983b-4c1b5fdbda1d None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:19.012362 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e275592-5531-4d2a-983b-4c1b5fdbda1d None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:19.013039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e275592-5531-4d2a-983b-4c1b5fdbda1d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:19.013343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e275592-5531-4d2a-983b-4c1b5fdbda1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:19.014989 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e275592-5531-4d2a-983b-4c1b5fdbda1d None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:19.022325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 575/1166] 10.222.0.70 () {56 vars in 1115 bytes} [Wed Aug 24 16:30:18 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:20.921444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6037357-0ad0-4398-ab83-c45a68c59af9 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:20.922059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6037357-0ad0-4398-ab83-c45a68c59af9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:20.922059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6037357-0ad0-4398-ab83-c45a68c59af9 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:20.923331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 592/1167] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:20.929436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c9cdd-fbfa-406f-b586-17b05f4029e2 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:20.929897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c9cdd-fbfa-406f-b586-17b05f4029e2 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:20.929998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c9cdd-fbfa-406f-b586-17b05f4029e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:20.948367 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-af5c9cdd-fbfa-406f-b586-17b05f4029e2 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:20.949589 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-af5c9cdd-fbfa-406f-b586-17b05f4029e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:21.019141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 576/1168] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:30:21.287527 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6037357-0ad0-4398-ab83-c45a68c59af9 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:21.347141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:21.392209 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6037357-0ad0-4398-ab83-c45a68c59af9 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:21.428842 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4d3ff-d58f-4220-b6ea-11a923839606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:21.429457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4d3ff-d58f-4220-b6ea-11a923839606 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:21.429731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4d3ff-d58f-4220-b6ea-11a923839606 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:21.429999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4d3ff-d58f-4220-b6ea-11a923839606 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:21.430666 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4d3ff-d58f-4220-b6ea-11a923839606 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:21.432099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4d3ff-d58f-4220-b6ea-11a923839606 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:21.512720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 593/1169] 10.222.0.70 () {60 vars in 1138 bytes} [Wed Aug 24 16:30:21 2022] POST /identity/v3/domains => generated 267 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:30:23.508397 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2101de-be38-4388-923a-bdbeeb64e38f None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:23.509205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2101de-be38-4388-923a-bdbeeb64e38f None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:23.509205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2101de-be38-4388-923a-bdbeeb64e38f None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:23.510340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 577/1170] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:23.516174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc213d2-1d17-4958-b486-1a23caccf576 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:23.516735 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc213d2-1d17-4958-b486-1a23caccf576 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:23.516735 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc213d2-1d17-4958-b486-1a23caccf576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:23.537712 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8cc213d2-1d17-4958-b486-1a23caccf576 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:23.539452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8cc213d2-1d17-4958-b486-1a23caccf576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:23.615334 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 594/1171] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:30:23.884907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2101de-be38-4388-923a-bdbeeb64e38f None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:23.908726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:24.009741 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2101de-be38-4388-923a-bdbeeb64e38f None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:24.051875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f1fa4-e561-4726-a666-46dc4a0080d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:24.052616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f1fa4-e561-4726-a666-46dc4a0080d9 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:24.052944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f1fa4-e561-4726-a666-46dc4a0080d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:24.053263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f1fa4-e561-4726-a666-46dc4a0080d9 None admin] PATH_INFO: `/v3/domains/de0b4d2b7cb247c5a82e820dec0ee37e` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:24.065597 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f1fa4-e561-4726-a666-46dc4a0080d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=de0b4d2b7cb247c5a82e820dec0ee37e)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:24.067128 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f1fa4-e561-4726-a666-46dc4a0080d9 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:24.068921 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 578/1172] 10.222.0.70 () {56 vars in 1183 bytes} [Wed Aug 24 16:30:23 2022] GET /identity/v3/domains/de0b4d2b7cb247c5a82e820dec0ee37e => generated 267 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:24.075653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc213d2-1d17-4958-b486-1a23caccf576 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:24.128160 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:24.184835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc213d2-1d17-4958-b486-1a23caccf576 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:24.223164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae37cf8-6761-4e75-a0e1-91095880461b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:24.223857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae37cf8-6761-4e75-a0e1-91095880461b None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:24.224361 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae37cf8-6761-4e75-a0e1-91095880461b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:24.224742 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae37cf8-6761-4e75-a0e1-91095880461b None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:24.225472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae37cf8-6761-4e75-a0e1-91095880461b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:24.226967 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae37cf8-6761-4e75-a0e1-91095880461b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:24.321835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 595/1173] 10.222.0.70 () {60 vars in 1141 bytes} [Wed Aug 24 16:30:24 2022] POST /identity/v3/projects => generated 361 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:30:26.320035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc3653-f4fe-432a-8fd6-be6e69042ab5 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:26.320644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc3653-f4fe-432a-8fd6-be6e69042ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:26.320644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc3653-f4fe-432a-8fd6-be6e69042ab5 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:26.321935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 579/1174] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:30:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:26.327366 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00bf8c-9425-41b6-83a8-303f347aa52a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:26.327841 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00bf8c-9425-41b6-83a8-303f347aa52a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:26.327953 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00bf8c-9425-41b6-83a8-303f347aa52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:26.345900 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fd00bf8c-9425-41b6-83a8-303f347aa52a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:26.347081 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fd00bf8c-9425-41b6-83a8-303f347aa52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:26.419061 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 596/1175] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:30:26.685778 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc3653-f4fe-432a-8fd6-be6e69042ab5 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:26.782198 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:26.814083 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc3653-f4fe-432a-8fd6-be6e69042ab5 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:26.853473 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52e438-2d83-42cf-ab38-3eec36ce974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:26.854125 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52e438-2d83-42cf-ab38-3eec36ce974a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:26.854411 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52e438-2d83-42cf-ab38-3eec36ce974a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:26.854688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52e438-2d83-42cf-ab38-3eec36ce974a None admin] PATH_INFO: `/v3/domains/de0b4d2b7cb247c5a82e820dec0ee37e` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:26.855874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52e438-2d83-42cf-ab38-3eec36ce974a None admin] RBAC: Authorizing `identity:get_domain(domain_id=de0b4d2b7cb247c5a82e820dec0ee37e)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:26.857370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52e438-2d83-42cf-ab38-3eec36ce974a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:26.859001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 580/1176] 10.222.0.70 () {56 vars in 1183 bytes} [Wed Aug 24 16:30:26 2022] GET /identity/v3/domains/de0b4d2b7cb247c5a82e820dec0ee37e => generated 267 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:30:26.868099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00bf8c-9425-41b6-83a8-303f347aa52a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:26.902346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:26.984334 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00bf8c-9425-41b6-83a8-303f347aa52a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:27.021401 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6f318-66ad-4ec0-a6c7-a7078643e87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:27.022704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6f318-66ad-4ec0-a6c7-a7078643e87f None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:27.023081 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6f318-66ad-4ec0-a6c7-a7078643e87f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:27.023528 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6f318-66ad-4ec0-a6c7-a7078643e87f None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:27.024271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6f318-66ad-4ec0-a6c7-a7078643e87f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:27.025855 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6f318-66ad-4ec0-a6c7-a7078643e87f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:27.115819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 597/1177] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:30:26 2022] POST /identity/v3/users => generated 311 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:30:29.118761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce6726-ab81-4a26-8036-95045c776a8d None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:29.119332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce6726-ab81-4a26-8036-95045c776a8d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:29.119332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce6726-ab81-4a26-8036-95045c776a8d None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:29.120707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 581/1178] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:29.126404 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6377693-efde-4aa0-a6a4-aa8e95d0f58a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:29.126885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6377693-efde-4aa0-a6a4-aa8e95d0f58a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:29.126986 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6377693-efde-4aa0-a6a4-aa8e95d0f58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:29.146013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e6377693-efde-4aa0-a6a4-aa8e95d0f58a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:29.147762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e6377693-efde-4aa0-a6a4-aa8e95d0f58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:29.219661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 598/1179] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:30:29.494214 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce6726-ab81-4a26-8036-95045c776a8d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:29.606874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce6726-ab81-4a26-8036-95045c776a8d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:29.644263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66bdc7-b0f1-4af5-8c6d-deb5db7be424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:29.645032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66bdc7-b0f1-4af5-8c6d-deb5db7be424 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:29.645319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66bdc7-b0f1-4af5-8c6d-deb5db7be424 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:29.645631 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66bdc7-b0f1-4af5-8c6d-deb5db7be424 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:29.658143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66bdc7-b0f1-4af5-8c6d-deb5db7be424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-bd66bdc7-b0f1-4af5-8c6d-deb5db7be424 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:30:29.662959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:29.664156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:29.665146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:29.665146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:29.665146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:29.665146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:29.665146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:30:29.665146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:30:29.665146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:30:29.665146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:30:29.665146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:30:29.665146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:30:29.665146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:30:29.665146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:30:29.665146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 582/1180] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:30:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:30:29.670866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6377693-efde-4aa0-a6a4-aa8e95d0f58a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:29.687850 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:29.855436 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6377693-efde-4aa0-a6a4-aa8e95d0f58a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:29.911537 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fd5be-8fe4-4234-802c-2360c9d32aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:29.912542 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fd5be-8fe4-4234-802c-2360c9d32aaa None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:29.912979 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fd5be-8fe4-4234-802c-2360c9d32aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:29.913407 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fd5be-8fe4-4234-802c-2360c9d32aaa None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:29.914447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fd5be-8fe4-4234-802c-2360c9d32aaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:29.914918 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fd5be-8fe4-4234-802c-2360c9d32aaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:29.917284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fd5be-8fe4-4234-802c-2360c9d32aaa None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:29.931553 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 599/1181] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:30:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:29.939181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66bdc7-b0f1-4af5-8c6d-deb5db7be424 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:29.943252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:30.064714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66bdc7-b0f1-4af5-8c6d-deb5db7be424 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:30.101652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bc764-d8b5-405f-a4cf-a7bd2f6abccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:30.102303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bc764-d8b5-405f-a4cf-a7bd2f6abccf None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:30.102634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bc764-d8b5-405f-a4cf-a7bd2f6abccf None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:30.102910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bc764-d8b5-405f-a4cf-a7bd2f6abccf None admin] PATH_INFO: `/v3/users/17fb466b4104462fb92b0cedb79440d5` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:30.120643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5bc764-d8b5-405f-a4cf-a7bd2f6abccf None admin] RBAC: Authorizing `identity:get_user(user_id=17fb466b4104462fb92b0cedb79440d5)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:30.121843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5bc764-d8b5-405f-a4cf-a7bd2f6abccf None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:30.123287 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 583/1182] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:30:29 2022] GET /identity/v3/users/17fb466b4104462fb92b0cedb79440d5 => generated 311 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:30.129330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fd5be-8fe4-4234-802c-2360c9d32aaa None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:30.210242 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:30.236983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fd5be-8fe4-4234-802c-2360c9d32aaa None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:30.276961 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c11c6-2e6b-427f-b621-cabf6e65fa0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:30.277578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c11c6-2e6b-427f-b621-cabf6e65fa0d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:30.277858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c11c6-2e6b-427f-b621-cabf6e65fa0d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:30.278130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c11c6-2e6b-427f-b621-cabf6e65fa0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:30.352825 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c11c6-2e6b-427f-b621-cabf6e65fa0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:30.354301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c11c6-2e6b-427f-b621-cabf6e65fa0d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:30.434107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:30.460626 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 600/1183] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:30:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:30:30.467159 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bc764-d8b5-405f-a4cf-a7bd2f6abccf None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:30.536651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:30.578226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bc764-d8b5-405f-a4cf-a7bd2f6abccf None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:30.627691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d820d65-eb67-43dd-9a5f-df5d3144638f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:30.628344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d820d65-eb67-43dd-9a5f-df5d3144638f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:30.628658 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d820d65-eb67-43dd-9a5f-df5d3144638f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:30.628937 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d820d65-eb67-43dd-9a5f-df5d3144638f None admin] PATH_INFO: `/v3/projects/d1c60a8b806445aa94c715d968444859` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:30.630105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d820d65-eb67-43dd-9a5f-df5d3144638f None admin] RBAC: Authorizing `identity:get_project(project_id=d1c60a8b806445aa94c715d968444859)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:30.631598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d820d65-eb67-43dd-9a5f-df5d3144638f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:30.633320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 584/1184] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:30:30 2022] GET /identity/v3/projects/d1c60a8b806445aa94c715d968444859 => generated 361 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:30.642859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c11c6-2e6b-427f-b621-cabf6e65fa0d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:30.757598 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c11c6-2e6b-427f-b621-cabf6e65fa0d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:30.795025 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8095527-4611-4042-94e1-e8377a78ea12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:30.795728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8095527-4611-4042-94e1-e8377a78ea12 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:30.796055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8095527-4611-4042-94e1-e8377a78ea12 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:30.796376 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8095527-4611-4042-94e1-e8377a78ea12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:30.797217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8095527-4611-4042-94e1-e8377a78ea12 None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=d1c60a8b806445aa94c715d968444859, user.id=17fb466b4104462fb92b0cedb79440d5) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:30.797571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8095527-4611-4042-94e1-e8377a78ea12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:30.799079 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8095527-4611-4042-94e1-e8377a78ea12 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:30.806166 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 601/1185] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:30:30 2022] GET /identity/v3/role_assignments?user.id=17fb466b4104462fb92b0cedb79440d5&scope.project.id=d1c60a8b806445aa94c715d968444859&role.id=e7f48343fc1542979ab4459865109e30 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:32.767528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c839ea-1b1d-4692-84e9-d563e99e75a8 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:32.768076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c839ea-1b1d-4692-84e9-d563e99e75a8 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:32.768076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c839ea-1b1d-4692-84e9-d563e99e75a8 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:32.769548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 585/1186] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:32.774889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d87bb-460c-4689-a642-e53ffc76be12 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:32.775308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d87bb-460c-4689-a642-e53ffc76be12 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:32.775452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d87bb-460c-4689-a642-e53ffc76be12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:32.793861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a57d87bb-460c-4689-a642-e53ffc76be12 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:32.794982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a57d87bb-460c-4689-a642-e53ffc76be12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:32.824703 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:32.861718 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 602/1187] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:30:33.133971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c839ea-1b1d-4692-84e9-d563e99e75a8 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:33.253767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c839ea-1b1d-4692-84e9-d563e99e75a8 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:33.293573 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:33.300214 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032811d2-5d45-49b8-87bc-20c712406f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:33.301001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032811d2-5d45-49b8-87bc-20c712406f98 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:33.310437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032811d2-5d45-49b8-87bc-20c712406f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:33.310437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032811d2-5d45-49b8-87bc-20c712406f98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:33.314051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032811d2-5d45-49b8-87bc-20c712406f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-032811d2-5d45-49b8-87bc-20c712406f98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:30:33.318978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:33.320147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:33.321116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:33.321116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:33.321116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:33.321116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:33.321116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:30:33.321116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:30:33.321116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:30:33.321116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:30:33.321116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:30:33.321116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:30:33.321116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:30:33.321116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:30:33.321116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 586/1188] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:30:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:30:33.326326 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d87bb-460c-4689-a642-e53ffc76be12 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:33.445076 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d87bb-460c-4689-a642-e53ffc76be12 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:33.483240 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b3731-f3ec-4514-95e8-d539c4665d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:33.483875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b3731-f3ec-4514-95e8-d539c4665d6d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:33.484155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b3731-f3ec-4514-95e8-d539c4665d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:33.484455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b3731-f3ec-4514-95e8-d539c4665d6d None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:33.485153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b3731-f3ec-4514-95e8-d539c4665d6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:33.485458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b3731-f3ec-4514-95e8-d539c4665d6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:33.486844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b3731-f3ec-4514-95e8-d539c4665d6d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:33.496883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 603/1189] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:30:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:33.503085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032811d2-5d45-49b8-87bc-20c712406f98 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:33.618355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032811d2-5d45-49b8-87bc-20c712406f98 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:33.659144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a9055-962a-47bc-9fed-057c0b62ee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:33.659783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a9055-962a-47bc-9fed-057c0b62ee1f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:33.660065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a9055-962a-47bc-9fed-057c0b62ee1f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:33.660339 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a9055-962a-47bc-9fed-057c0b62ee1f None admin] PATH_INFO: `/v3/users/17fb466b4104462fb92b0cedb79440d5` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:33.661536 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a9055-962a-47bc-9fed-057c0b62ee1f None admin] RBAC: Authorizing `identity:get_user(user_id=17fb466b4104462fb92b0cedb79440d5)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:33.664141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a9055-962a-47bc-9fed-057c0b62ee1f None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:33.665877 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 587/1190] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:30:33 2022] GET /identity/v3/users/17fb466b4104462fb92b0cedb79440d5 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:33.672098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b3731-f3ec-4514-95e8-d539c4665d6d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:33.710230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:33.786827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b3731-f3ec-4514-95e8-d539c4665d6d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:33.820596 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e71e5-3fa1-47e3-be3b-02da9b864473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:33.821252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e71e5-3fa1-47e3-be3b-02da9b864473 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:33.821530 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e71e5-3fa1-47e3-be3b-02da9b864473 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:33.821803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e71e5-3fa1-47e3-be3b-02da9b864473 None admin] PATH_INFO: `/v3/projects/d1c60a8b806445aa94c715d968444859` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:33.822915 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e71e5-3fa1-47e3-be3b-02da9b864473 None admin] RBAC: Authorizing `identity:get_project(project_id=d1c60a8b806445aa94c715d968444859)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:33.824352 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e71e5-3fa1-47e3-be3b-02da9b864473 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:33.826015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 604/1191] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:30:33 2022] GET /identity/v3/projects/d1c60a8b806445aa94c715d968444859 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:30:33.832048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a9055-962a-47bc-9fed-057c0b62ee1f None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:33.893260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:33.938686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a9055-962a-47bc-9fed-057c0b62ee1f None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:33.975123 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7935fee-4de6-4ac7-abbe-56f7481d89a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:33.975712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7935fee-4de6-4ac7-abbe-56f7481d89a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:33.978139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7935fee-4de6-4ac7-abbe-56f7481d89a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:33.978139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7935fee-4de6-4ac7-abbe-56f7481d89a8 None admin] PATH_INFO: `/v3/projects/d1c60a8b806445aa94c715d968444859/users/17fb466b4104462fb92b0cedb79440d5/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:33.978346 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7935fee-4de6-4ac7-abbe-56f7481d89a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1c60a8b806445aa94c715d968444859, user_id=17fb466b4104462fb92b0cedb79440d5, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:33.979756 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7935fee-4de6-4ac7-abbe-56f7481d89a8 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:34.046792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 588/1192] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:30:33 2022] PUT /identity/v3/projects/d1c60a8b806445aa94c715d968444859/users/17fb466b4104462fb92b0cedb79440d5/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:30:35.965813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81cc59-0512-4eb1-b89e-27158ddb2ec4 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:35.966401 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81cc59-0512-4eb1-b89e-27158ddb2ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:35.966401 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81cc59-0512-4eb1-b89e-27158ddb2ec4 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:35.967704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 605/1193] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:35.973571 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c80c6-cee9-4a87-ac6f-c81dfd0c6e9f None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:35.974054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c80c6-cee9-4a87-ac6f-c81dfd0c6e9f None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:35.974145 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c80c6-cee9-4a87-ac6f-c81dfd0c6e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:35.993186 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-1a8c80c6-cee9-4a87-ac6f-c81dfd0c6e9f None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:35.994394 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1a8c80c6-cee9-4a87-ac6f-c81dfd0c6e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:36.068020 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 589/1194] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:30:36.336236 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81cc59-0512-4eb1-b89e-27158ddb2ec4 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:36.434157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:36.438849 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81cc59-0512-4eb1-b89e-27158ddb2ec4 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:36.476329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71b130-5a2f-49da-a05e-11b00c89805a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:36.477037 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71b130-5a2f-49da-a05e-11b00c89805a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:36.477312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71b130-5a2f-49da-a05e-11b00c89805a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:36.477578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71b130-5a2f-49da-a05e-11b00c89805a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:36.489449 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71b130-5a2f-49da-a05e-11b00c89805a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-6a71b130-5a2f-49da-a05e-11b00c89805a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:30:36.494894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:36.496062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:36.497144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:36.497144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:36.497144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:36.497144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:36.497144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:30:36.497144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:30:36.497144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:30:36.497144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:30:36.497144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:30:36.497144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:30:36.497144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:30:36.497144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:30:36.497144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 606/1195] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:30:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:30:36.502253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c80c6-cee9-4a87-ac6f-c81dfd0c6e9f None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:36.554317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:36.611042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c80c6-cee9-4a87-ac6f-c81dfd0c6e9f None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:36.647878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31955f8-07be-4d6b-a9e2-d33cf6bf72a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:36.648593 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31955f8-07be-4d6b-a9e2-d33cf6bf72a9 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:36.648927 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31955f8-07be-4d6b-a9e2-d33cf6bf72a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:36.649238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31955f8-07be-4d6b-a9e2-d33cf6bf72a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:36.649947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31955f8-07be-4d6b-a9e2-d33cf6bf72a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:36.650286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31955f8-07be-4d6b-a9e2-d33cf6bf72a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:36.651699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31955f8-07be-4d6b-a9e2-d33cf6bf72a9 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:36.660381 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 590/1196] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:30:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:36.666966 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71b130-5a2f-49da-a05e-11b00c89805a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:36.767403 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71b130-5a2f-49da-a05e-11b00c89805a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:36.802286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c41d4-eb53-4e72-b4b1-8632bab97476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:36.802945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c41d4-eb53-4e72-b4b1-8632bab97476 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:36.803223 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c41d4-eb53-4e72-b4b1-8632bab97476 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:36.803496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c41d4-eb53-4e72-b4b1-8632bab97476 None admin] PATH_INFO: `/v3/users/17fb466b4104462fb92b0cedb79440d5` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:36.804803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c41d4-eb53-4e72-b4b1-8632bab97476 None admin] RBAC: Authorizing `identity:get_user(user_id=17fb466b4104462fb92b0cedb79440d5)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:36.806231 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c41d4-eb53-4e72-b4b1-8632bab97476 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:36.807802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 607/1197] 10.222.0.70 () {56 vars in 1177 bytes} [Wed Aug 24 16:30:36 2022] GET /identity/v3/users/17fb466b4104462fb92b0cedb79440d5 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:36.813992 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31955f8-07be-4d6b-a9e2-d33cf6bf72a9 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:36.915199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31955f8-07be-4d6b-a9e2-d33cf6bf72a9 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:36.925554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:36.950085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1813fe-17fa-4318-a1c0-9237a7c74830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:36.950696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1813fe-17fa-4318-a1c0-9237a7c74830 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:36.950978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1813fe-17fa-4318-a1c0-9237a7c74830 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:36.951250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1813fe-17fa-4318-a1c0-9237a7c74830 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:37.017940 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1813fe-17fa-4318-a1c0-9237a7c74830 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:37.019391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1813fe-17fa-4318-a1c0-9237a7c74830 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:37.121235 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 591/1198] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:30:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:30:37.127229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c41d4-eb53-4e72-b4b1-8632bab97476 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:37.161692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:37.250290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c41d4-eb53-4e72-b4b1-8632bab97476 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:37.289636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceed98f-cf1d-421c-8683-8f1d1775ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:37.290273 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceed98f-cf1d-421c-8683-8f1d1775ddf4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:37.290552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceed98f-cf1d-421c-8683-8f1d1775ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:37.290820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceed98f-cf1d-421c-8683-8f1d1775ddf4 None admin] PATH_INFO: `/v3/projects/d1c60a8b806445aa94c715d968444859` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:37.291999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceed98f-cf1d-421c-8683-8f1d1775ddf4 None admin] RBAC: Authorizing `identity:get_project(project_id=d1c60a8b806445aa94c715d968444859)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:37.293472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceed98f-cf1d-421c-8683-8f1d1775ddf4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:37.295122 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 608/1199] 10.222.0.70 () {56 vars in 1186 bytes} [Wed Aug 24 16:30:37 2022] GET /identity/v3/projects/d1c60a8b806445aa94c715d968444859 => generated 361 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:37.301556 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1813fe-17fa-4318-a1c0-9237a7c74830 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:37.319516 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:37.425316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1813fe-17fa-4318-a1c0-9237a7c74830 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:37.473306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25f2a4-5924-47ba-96d9-ae63038c356b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:37.473949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25f2a4-5924-47ba-96d9-ae63038c356b None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:37.474233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25f2a4-5924-47ba-96d9-ae63038c356b None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:37.474508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25f2a4-5924-47ba-96d9-ae63038c356b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:37.475231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25f2a4-5924-47ba-96d9-ae63038c356b None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=d1c60a8b806445aa94c715d968444859, user.id=17fb466b4104462fb92b0cedb79440d5) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:37.475538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25f2a4-5924-47ba-96d9-ae63038c356b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:37.477103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25f2a4-5924-47ba-96d9-ae63038c356b None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:37.483115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 592/1200] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:30:37 2022] GET /identity/v3/role_assignments?user.id=17fb466b4104462fb92b0cedb79440d5&scope.project.id=d1c60a8b806445aa94c715d968444859&role.id=e7f48343fc1542979ab4459865109e30 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:30:47.285470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=53 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:47.480379 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=63 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:53.039938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ac93b-c399-4b9d-adf0-ed17098620e4 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:53.040608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ac93b-c399-4b9d-adf0-ed17098620e4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:53.040608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ac93b-c399-4b9d-adf0-ed17098620e4 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:53.041898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 609/1201] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:31:53.047255 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a481e-17f5-4f83-b24b-e442a4a09c52 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:53.047703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a481e-17f5-4f83-b24b-e442a4a09c52 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:53.047805 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a481e-17f5-4f83-b24b-e442a4a09c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:53.066154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-064a481e-17f5-4f83-b24b-e442a4a09c52 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:53.067350 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-064a481e-17f5-4f83-b24b-e442a4a09c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:53.134648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 593/1202] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:31:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:31:53.406368 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ac93b-c399-4b9d-adf0-ed17098620e4 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:53.509456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ac93b-c399-4b9d-adf0-ed17098620e4 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:53.543614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83997695-6340-47c7-a401-a3fcfdf90405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:53.544274 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83997695-6340-47c7-a401-a3fcfdf90405 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:53.544582 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83997695-6340-47c7-a401-a3fcfdf90405 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:53.544859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83997695-6340-47c7-a401-a3fcfdf90405 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:53.555727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83997695-6340-47c7-a401-a3fcfdf90405 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:53.557198 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83997695-6340-47c7-a401-a3fcfdf90405 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-83997695-6340-47c7-a401-a3fcfdf90405 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:31:53.572011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:31:53.578452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:31:53.579506 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:31:53.579506 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:31:53.579506 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:31:53.579506 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 610/1203] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:31:53.579761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a481e-17f5-4f83-b24b-e442a4a09c52 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:53.680035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a481e-17f5-4f83-b24b-e442a4a09c52 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:53.708837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:53.714678 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca927f-33c3-4d0b-911f-68e942505c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:53.715277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca927f-33c3-4d0b-911f-68e942505c6d None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:53.715556 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca927f-33c3-4d0b-911f-68e942505c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:53.715866 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca927f-33c3-4d0b-911f-68e942505c6d None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:53.716539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca927f-33c3-4d0b-911f-68e942505c6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:31:53.716836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca927f-33c3-4d0b-911f-68e942505c6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:53.718187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca927f-33c3-4d0b-911f-68e942505c6d None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:53.729136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 594/1204] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:31:53.735508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83997695-6340-47c7-a401-a3fcfdf90405 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:53.788786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:53.835251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83997695-6340-47c7-a401-a3fcfdf90405 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:53.867628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9389e3-ba67-4a73-b5ca-e72bc92b0191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:53.868231 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9389e3-ba67-4a73-b5ca-e72bc92b0191 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:53.868538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9389e3-ba67-4a73-b5ca-e72bc92b0191 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:53.875415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9389e3-ba67-4a73-b5ca-e72bc92b0191 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:53.875415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9389e3-ba67-4a73-b5ca-e72bc92b0191 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:53.875415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9389e3-ba67-4a73-b5ca-e72bc92b0191 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:54.060898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 611/1205] 10.222.0.70 () {60 vars in 1132 bytes} [Wed Aug 24 16:31:53 2022] POST /identity/v3/users => generated 251 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:31:56.020052 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594163f-abbc-4356-bd85-e4949c6371f6 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:56.020628 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594163f-abbc-4356-bd85-e4949c6371f6 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:56.020628 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594163f-abbc-4356-bd85-e4949c6371f6 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:56.021986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 595/1206] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:31:56.027174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a69a6-009b-49cc-8a29-ff4c5db4d44a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:56.027599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a69a6-009b-49cc-8a29-ff4c5db4d44a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:56.027699 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a69a6-009b-49cc-8a29-ff4c5db4d44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:56.046501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0f4a69a6-009b-49cc-8a29-ff4c5db4d44a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:56.055064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0f4a69a6-009b-49cc-8a29-ff4c5db4d44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:56.115298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 612/1207] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:31:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:31:56.391685 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594163f-abbc-4356-bd85-e4949c6371f6 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:56.494517 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594163f-abbc-4356-bd85-e4949c6371f6 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:56.527673 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2b7d4-cdd8-429e-87b1-bbb6e4ffafe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:56.528299 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2b7d4-cdd8-429e-87b1-bbb6e4ffafe6 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:56.528611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2b7d4-cdd8-429e-87b1-bbb6e4ffafe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:56.528883 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2b7d4-cdd8-429e-87b1-bbb6e4ffafe6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:56.540065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2b7d4-cdd8-429e-87b1-bbb6e4ffafe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-50c2b7d4-cdd8-429e-87b1-bbb6e4ffafe6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:31:56.544783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:31:56.545925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:31:56.546997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:31:56.546997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:31:56.546997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:31:56.546997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:31:56.546997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:31:56.546997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:31:56.546997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:31:56.546997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:31:56.546997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:31:56.546997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:31:56.546997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:31:56.546997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:31:56.546997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 596/1208] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:31:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:31:56.553443 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a69a6-009b-49cc-8a29-ff4c5db4d44a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:56.592655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:56.653837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a69a6-009b-49cc-8a29-ff4c5db4d44a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:56.687594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b1a6f-7178-4ad1-8e8c-7faa7af0a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:56.688629 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b1a6f-7178-4ad1-8e8c-7faa7af0a5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:56.688629 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b1a6f-7178-4ad1-8e8c-7faa7af0a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:56.688873 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b1a6f-7178-4ad1-8e8c-7faa7af0a5e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:56.689483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b1a6f-7178-4ad1-8e8c-7faa7af0a5e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:31:56.689790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b1a6f-7178-4ad1-8e8c-7faa7af0a5e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:56.691166 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b1a6f-7178-4ad1-8e8c-7faa7af0a5e9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:56.699105 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 613/1209] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:31:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:31:56.705177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2b7d4-cdd8-429e-87b1-bbb6e4ffafe6 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:56.756543 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:56.806187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2b7d4-cdd8-429e-87b1-bbb6e4ffafe6 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:56.839852 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cea57f-cff5-4400-836f-03d6e33a6e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:56.840509 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cea57f-cff5-4400-836f-03d6e33a6e14 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:56.840795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cea57f-cff5-4400-836f-03d6e33a6e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:56.841105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cea57f-cff5-4400-836f-03d6e33a6e14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:56.852389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cea57f-cff5-4400-836f-03d6e33a6e14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:56.853870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cea57f-cff5-4400-836f-03d6e33a6e14 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-b3cea57f-cff5-4400-836f-03d6e33a6e14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:31:56.867834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:31:56.869012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:31:56.869941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:31:56.869941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:31:56.869941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:31:56.869941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 597/1210] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:31:56.875179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b1a6f-7178-4ad1-8e8c-7faa7af0a5e9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:56.974911 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:56.977691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b1a6f-7178-4ad1-8e8c-7faa7af0a5e9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:57.010266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4e69-fa9c-47f1-a580-08a09d669f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:57.010863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4e69-fa9c-47f1-a580-08a09d669f14 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:57.011140 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4e69-fa9c-47f1-a580-08a09d669f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:57.011668 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4e69-fa9c-47f1-a580-08a09d669f14 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:57.012316 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c4e69-fa9c-47f1-a580-08a09d669f14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:31:57.012645 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c4e69-fa9c-47f1-a580-08a09d669f14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:57.014004 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c4e69-fa9c-47f1-a580-08a09d669f14 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:57.025751 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 614/1211] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:31:57.031617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cea57f-cff5-4400-836f-03d6e33a6e14 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:57.133308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cea57f-cff5-4400-836f-03d6e33a6e14 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:57.134849 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:57.167811 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7520a-f6e7-44cc-b72c-cef7c9708d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:57.172152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7520a-f6e7-44cc-b72c-cef7c9708d7a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:57.172152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7520a-f6e7-44cc-b72c-cef7c9708d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:57.172152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7520a-f6e7-44cc-b72c-cef7c9708d7a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:57.176715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7520a-f6e7-44cc-b72c-cef7c9708d7a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:57.178186 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7520a-f6e7-44cc-b72c-cef7c9708d7a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-1ec7520a-f6e7-44cc-b72c-cef7c9708d7a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:31:57.189297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:31:57.191314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:31:57.192609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:31:57.192609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:31:57.192609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:31:57.192609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:31:57.192609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:31:57.192609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:31:57.192609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:31:57.192609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:31:57.192609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:31:57.192609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:31:57.192609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 598/1212] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:31:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:31:57.196973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4e69-fa9c-47f1-a580-08a09d669f14 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:57.295827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4e69-fa9c-47f1-a580-08a09d669f14 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:57.330313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faab90c-6106-4004-ac31-3263f68abdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:57.330490 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faab90c-6106-4004-ac31-3263f68abdf2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:57.330731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faab90c-6106-4004-ac31-3263f68abdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:57.331557 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faab90c-6106-4004-ac31-3263f68abdf2 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:57.332291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faab90c-6106-4004-ac31-3263f68abdf2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:31:57.332659 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faab90c-6106-4004-ac31-3263f68abdf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:57.334078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faab90c-6106-4004-ac31-3263f68abdf2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:57.351029 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 615/1213] 10.222.0.70 () {56 vars in 1143 bytes} [Wed Aug 24 16:31:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:31:57.356984 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7520a-f6e7-44cc-b72c-cef7c9708d7a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:57.466891 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7520a-f6e7-44cc-b72c-cef7c9708d7a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:57.500837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5e899-57c6-45f9-b4e9-b1180fc5f437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:57.501440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5e899-57c6-45f9-b4e9-b1180fc5f437 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:57.501718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5e899-57c6-45f9-b4e9-b1180fc5f437 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:57.501989 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5e899-57c6-45f9-b4e9-b1180fc5f437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:57.532195 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:57.578932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5e899-57c6-45f9-b4e9-b1180fc5f437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:57.578932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5e899-57c6-45f9-b4e9-b1180fc5f437 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:57.671247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 599/1214] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:31:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:31:57.677765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faab90c-6106-4004-ac31-3263f68abdf2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:57.700350 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:57.788478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faab90c-6106-4004-ac31-3263f68abdf2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:57.823314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ac23f-e5bb-4862-a89b-8323d78ae085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:57.823984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ac23f-e5bb-4862-a89b-8323d78ae085 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:57.824260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ac23f-e5bb-4862-a89b-8323d78ae085 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:57.824591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ac23f-e5bb-4862-a89b-8323d78ae085 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:57.834870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ac23f-e5bb-4862-a89b-8323d78ae085 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:57.836233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ac23f-e5bb-4862-a89b-8323d78ae085 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-271ac23f-e5bb-4862-a89b-8323d78ae085 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:31:57.851328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:31:57.852541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:31:57.853602 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:31:57.853602 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:31:57.853602 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:31:57.853602 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 616/1215] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:31:57.858356 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5e899-57c6-45f9-b4e9-b1180fc5f437 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:57.919854 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:57.959477 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5e899-57c6-45f9-b4e9-b1180fc5f437 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:57.994261 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c67e1-968d-4fbd-aa2d-a0c3f89cd523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:57.994858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c67e1-968d-4fbd-aa2d-a0c3f89cd523 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:57.995133 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c67e1-968d-4fbd-aa2d-a0c3f89cd523 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:57.995441 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c67e1-968d-4fbd-aa2d-a0c3f89cd523 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:57.996074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c67e1-968d-4fbd-aa2d-a0c3f89cd523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:31:57.996370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c67e1-968d-4fbd-aa2d-a0c3f89cd523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:57.997750 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c67e1-968d-4fbd-aa2d-a0c3f89cd523 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:58.008889 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 600/1216] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:31:58.015032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ac23f-e5bb-4862-a89b-8323d78ae085 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:58.086368 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:58.122133 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ac23f-e5bb-4862-a89b-8323d78ae085 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:58.155318 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e00f5-2371-466e-a9d7-0a07c6d457fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:58.155951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e00f5-2371-466e-a9d7-0a07c6d457fa None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:58.156229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e00f5-2371-466e-a9d7-0a07c6d457fa None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:58.156532 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e00f5-2371-466e-a9d7-0a07c6d457fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:58.163558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e00f5-2371-466e-a9d7-0a07c6d457fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:58.165023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e00f5-2371-466e-a9d7-0a07c6d457fa None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:58.176517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-203e00f5-2371-466e-a9d7-0a07c6d457fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:31:58.176517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:31:58.176517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:31:58.176517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:31:58.176517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:31:58.176517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:31:58.176517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:31:58.176517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:31:58.176517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:31:58.176517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:31:58.176517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:31:58.176517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:31:58.176517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:31:58.176517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:31:58.176517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:31:58.177548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:31:58.178650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:31:58.178650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:31:58.178650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:31:58.178650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:31:58.178650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:31:58.178650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:31:58.178650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:31:58.178650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:31:58.178650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 617/1217] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:31:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:31:58.183620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c67e1-968d-4fbd-aa2d-a0c3f89cd523 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:58.286463 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c67e1-968d-4fbd-aa2d-a0c3f89cd523 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:58.309102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:58.320947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2131fe-e74b-4928-afd9-e06e27934068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:58.321690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2131fe-e74b-4928-afd9-e06e27934068 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:58.322085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2131fe-e74b-4928-afd9-e06e27934068 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:58.322386 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2131fe-e74b-4928-afd9-e06e27934068 None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:58.323136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2131fe-e74b-4928-afd9-e06e27934068 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:31:58.323442 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2131fe-e74b-4928-afd9-e06e27934068 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:58.324900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2131fe-e74b-4928-afd9-e06e27934068 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:58.338197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 601/1218] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:31:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:31:58.344603 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e00f5-2371-466e-a9d7-0a07c6d457fa None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:58.445266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e00f5-2371-466e-a9d7-0a07c6d457fa None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:58.474691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:58.481665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725014b-422f-4e70-b113-b3ee2ddf2536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:58.482274 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725014b-422f-4e70-b113-b3ee2ddf2536 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:58.482552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725014b-422f-4e70-b113-b3ee2ddf2536 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:58.482824 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725014b-422f-4e70-b113-b3ee2ddf2536 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:58.483549 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725014b-422f-4e70-b113-b3ee2ddf2536 None admin] RBAC: Adding query filter params (role.id=bb34a44869834493b10a99cc751ab6c2, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=fc4c43c85a5d49979ba803fdb1041c4e) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:31:58.483856 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725014b-422f-4e70-b113-b3ee2ddf2536 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:58.485301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725014b-422f-4e70-b113-b3ee2ddf2536 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:58.490655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 618/1219] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:31:58 2022] GET /identity/v3/role_assignments?user.id=fc4c43c85a5d49979ba803fdb1041c4e&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=bb34a44869834493b10a99cc751ab6c2 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:00.417606 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6fcc7-89cb-4406-873b-cb3315b2bfa9 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:00.418249 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6fcc7-89cb-4406-873b-cb3315b2bfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:00.418249 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6fcc7-89cb-4406-873b-cb3315b2bfa9 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:00.419505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 602/1220] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:00.425811 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f0501-2b63-434b-9b44-b977481761a3 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:00.431239 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f0501-2b63-434b-9b44-b977481761a3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:00.431239 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f0501-2b63-434b-9b44-b977481761a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:00.444884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-1e3f0501-2b63-434b-9b44-b977481761a3 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:00.446102 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-1e3f0501-2b63-434b-9b44-b977481761a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:00.513283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 619/1221] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:32:00.779713 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6fcc7-89cb-4406-873b-cb3315b2bfa9 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:00.884123 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6fcc7-89cb-4406-873b-cb3315b2bfa9 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:00.919835 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a12452-c4f0-48dd-92df-e79213c08e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:00.920527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a12452-c4f0-48dd-92df-e79213c08e8f None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:00.920820 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a12452-c4f0-48dd-92df-e79213c08e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:00.921096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a12452-c4f0-48dd-92df-e79213c08e8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:00.931461 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a12452-c4f0-48dd-92df-e79213c08e8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-a8a12452-c4f0-48dd-92df-e79213c08e8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:32:00.936207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:00.938752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:00.939575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:00.939575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:00.939575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:00.939575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:00.939575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:32:00.939575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:32:00.939575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:32:00.939575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:32:00.939575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:32:00.939575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:32:00.939575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:32:00.939575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:00.939575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 603/1222] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:32:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:00.944115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f0501-2b63-434b-9b44-b977481761a3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:01.044242 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f0501-2b63-434b-9b44-b977481761a3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:01.068098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:01.078647 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b4d5f-6da8-4706-a797-d61a956aad6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:01.080348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b4d5f-6da8-4706-a797-d61a956aad6e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:01.080348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b4d5f-6da8-4706-a797-d61a956aad6e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:01.080348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b4d5f-6da8-4706-a797-d61a956aad6e None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:01.080872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b4d5f-6da8-4706-a797-d61a956aad6e None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:01.080872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b4d5f-6da8-4706-a797-d61a956aad6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:01.082145 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b4d5f-6da8-4706-a797-d61a956aad6e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:01.090056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 620/1223] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:32:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:01.095825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a12452-c4f0-48dd-92df-e79213c08e8f None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:01.141852 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:01.198259 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a12452-c4f0-48dd-92df-e79213c08e8f None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:01.232249 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b7de8-9b4f-4c85-8d9e-ea6a1b395721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:01.232914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b7de8-9b4f-4c85-8d9e-ea6a1b395721 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:01.233197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b7de8-9b4f-4c85-8d9e-ea6a1b395721 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:01.233471 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b7de8-9b4f-4c85-8d9e-ea6a1b395721 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:01.243250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b7de8-9b4f-4c85-8d9e-ea6a1b395721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:01.244724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b7de8-9b4f-4c85-8d9e-ea6a1b395721 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-ca8b7de8-9b4f-4c85-8d9e-ea6a1b395721 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:32:01.259332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:01.260493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:32:01.261927 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:32:01.261927 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:01.261927 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:01.261927 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 604/1224] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:01.266671 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b4d5f-6da8-4706-a797-d61a956aad6e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:01.363533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b4d5f-6da8-4706-a797-d61a956aad6e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:01.397494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edafff-a8d1-4b73-9513-ef451abbfa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:01.398096 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edafff-a8d1-4b73-9513-ef451abbfa0a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:01.398376 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edafff-a8d1-4b73-9513-ef451abbfa0a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:01.398648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edafff-a8d1-4b73-9513-ef451abbfa0a None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:01.399283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6edafff-a8d1-4b73-9513-ef451abbfa0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:01.399586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6edafff-a8d1-4b73-9513-ef451abbfa0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:01.400998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6edafff-a8d1-4b73-9513-ef451abbfa0a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:01.413131 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 621/1225] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:01.418931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b7de8-9b4f-4c85-8d9e-ea6a1b395721 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:01.511996 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:01.519106 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b7de8-9b4f-4c85-8d9e-ea6a1b395721 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:01.552188 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f05b2-1830-4120-a59d-f03d9f6891d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:01.552852 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f05b2-1830-4120-a59d-f03d9f6891d3 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:01.553131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f05b2-1830-4120-a59d-f03d9f6891d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:01.553398 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f05b2-1830-4120-a59d-f03d9f6891d3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:01.560811 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f05b2-1830-4120-a59d-f03d9f6891d3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:01.562239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f05b2-1830-4120-a59d-f03d9f6891d3 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-ae2f05b2-1830-4120-a59d-f03d9f6891d3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:32:01.574371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:01.575486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:01.576560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:01.576560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:01.576560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:32:01.576560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:32:01.576560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:32:01.576560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:32:01.576560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:32:01.576560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:32:01.576560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:32:01.576560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:01.576560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 605/1226] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:32:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:01.582066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edafff-a8d1-4b73-9513-ef451abbfa0a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:01.630675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:01.683353 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edafff-a8d1-4b73-9513-ef451abbfa0a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:01.717286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a8274-6b4b-4be9-8b4a-cc723e558374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:01.717894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a8274-6b4b-4be9-8b4a-cc723e558374 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:01.718186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a8274-6b4b-4be9-8b4a-cc723e558374 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:01.718449 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a8274-6b4b-4be9-8b4a-cc723e558374 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:01.719174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a8274-6b4b-4be9-8b4a-cc723e558374 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:01.719480 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a8274-6b4b-4be9-8b4a-cc723e558374 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:01.721029 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a8274-6b4b-4be9-8b4a-cc723e558374 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:01.737732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 622/1227] 10.222.0.70 () {56 vars in 1143 bytes} [Wed Aug 24 16:32:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:32:01.743814 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f05b2-1830-4120-a59d-f03d9f6891d3 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:01.844712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f05b2-1830-4120-a59d-f03d9f6891d3 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:01.878363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ebc15-031e-48d4-8160-c8a2450edc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:01.878997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ebc15-031e-48d4-8160-c8a2450edc0d None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:01.879277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ebc15-031e-48d4-8160-c8a2450edc0d None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:01.879554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ebc15-031e-48d4-8160-c8a2450edc0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:01.890585 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ebc15-031e-48d4-8160-c8a2450edc0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:01.891966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ebc15-031e-48d4-8160-c8a2450edc0d None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-122ebc15-031e-48d4-8160-c8a2450edc0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:32:01.910621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:01.914790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:32:01.915803 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:32:01.915803 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:01.915803 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:01.915803 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 606/1228] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:01.917831 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a8274-6b4b-4be9-8b4a-cc723e558374 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:02.017352 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a8274-6b4b-4be9-8b4a-cc723e558374 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:02.020179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:02.051635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b7581-35c0-44e6-89c6-e5050e7b3a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:02.052241 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b7581-35c0-44e6-89c6-e5050e7b3a3c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:02.052554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b7581-35c0-44e6-89c6-e5050e7b3a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:02.052831 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b7581-35c0-44e6-89c6-e5050e7b3a3c None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:02.053473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b7581-35c0-44e6-89c6-e5050e7b3a3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:02.053779 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b7581-35c0-44e6-89c6-e5050e7b3a3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:02.055148 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b7581-35c0-44e6-89c6-e5050e7b3a3c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:02.066218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 623/1229] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:02.071996 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ebc15-031e-48d4-8160-c8a2450edc0d None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:02.081861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:02.172949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ebc15-031e-48d4-8160-c8a2450edc0d None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:02.205244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e7d5b-3210-43a8-96d3-0e9ea8bb3f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:02.206444 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e7d5b-3210-43a8-96d3-0e9ea8bb3f95 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:02.206729 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e7d5b-3210-43a8-96d3-0e9ea8bb3f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:02.207003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e7d5b-3210-43a8-96d3-0e9ea8bb3f95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:02.213718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e7d5b-3210-43a8-96d3-0e9ea8bb3f95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:02.215154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e7d5b-3210-43a8-96d3-0e9ea8bb3f95 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:02.227239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-bd3e7d5b-3210-43a8-96d3-0e9ea8bb3f95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:32:02.227239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:02.227239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:02.227239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:02.227239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:02.227239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:02.227239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:02.227239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:02.227239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:02.227239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:02.227239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:02.227239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:02.227239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:32:02.227239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:32:02.227239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:02.228303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:02.229548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:32:02.229548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:32:02.229548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:32:02.229548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:32:02.229548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:32:02.229548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:32:02.229548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:32:02.229548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:02.229548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 607/1230] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:32:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:02.234616 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b7581-35c0-44e6-89c6-e5050e7b3a3c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:02.337214 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b7581-35c0-44e6-89c6-e5050e7b3a3c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:02.370307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ae5cf-7a0b-4f1a-9106-651962f2baec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:02.370905 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ae5cf-7a0b-4f1a-9106-651962f2baec None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:02.371185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ae5cf-7a0b-4f1a-9106-651962f2baec None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:02.371457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ae5cf-7a0b-4f1a-9106-651962f2baec None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:02.372140 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ae5cf-7a0b-4f1a-9106-651962f2baec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:02.372468 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ae5cf-7a0b-4f1a-9106-651962f2baec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:02.373888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ae5cf-7a0b-4f1a-9106-651962f2baec None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:02.386881 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 624/1231] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:32:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:02.393970 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e7d5b-3210-43a8-96d3-0e9ea8bb3f95 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:02.461053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:02.496003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e7d5b-3210-43a8-96d3-0e9ea8bb3f95 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:02.530140 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b60b5-f2a2-4a9f-a70f-c034453a6be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:02.530796 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b60b5-f2a2-4a9f-a70f-c034453a6be3 None admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:02.531078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b60b5-f2a2-4a9f-a70f-c034453a6be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:02.531355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b60b5-f2a2-4a9f-a70f-c034453a6be3 None admin] PATH_INFO: `/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/fc4c43c85a5d49979ba803fdb1041c4e/roles/bb34a44869834493b10a99cc751ab6c2` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:02.546499 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b60b5-f2a2-4a9f-a70f-c034453a6be3 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc920dce30734ac7a69dc039d8c74ce1, user_id=fc4c43c85a5d49979ba803fdb1041c4e, role_id=bb34a44869834493b10a99cc751ab6c2)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:02.547945 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b60b5-f2a2-4a9f-a70f-c034453a6be3 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:02.666011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 608/1232] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:32:02 2022] PUT /identity/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/fc4c43c85a5d49979ba803fdb1041c4e/roles/bb34a44869834493b10a99cc751ab6c2 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:32:04.558485 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112faa28-61a9-4ac8-824b-16b9ee5c273b None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:04.559024 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112faa28-61a9-4ac8-824b-16b9ee5c273b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:04.559134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112faa28-61a9-4ac8-824b-16b9ee5c273b None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:04.560517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 625/1233] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:04.566565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9732d6f-48ae-4bef-951a-80e0df76a923 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:04.572200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9732d6f-48ae-4bef-951a-80e0df76a923 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:04.572200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9732d6f-48ae-4bef-951a-80e0df76a923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:04.585789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-d9732d6f-48ae-4bef-951a-80e0df76a923 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:04.586983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d9732d6f-48ae-4bef-951a-80e0df76a923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:04.658662 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 609/1234] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:32:04.927701 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112faa28-61a9-4ac8-824b-16b9ee5c273b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:04.952966 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:05.026777 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112faa28-61a9-4ac8-824b-16b9ee5c273b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:05.060998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824f35e-06b2-4b27-9d83-82f2ab247678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:05.061619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824f35e-06b2-4b27-9d83-82f2ab247678 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:05.061926 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824f35e-06b2-4b27-9d83-82f2ab247678 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:05.062196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824f35e-06b2-4b27-9d83-82f2ab247678 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:05.072486 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824f35e-06b2-4b27-9d83-82f2ab247678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-b824f35e-06b2-4b27-9d83-82f2ab247678 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:32:05.077045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:05.082504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:05.083471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:05.083471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:05.083471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:05.083471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:05.083471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:32:05.083471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:32:05.083471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:32:05.083471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:32:05.083471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:32:05.083471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:32:05.083471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:32:05.083471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:32:05.083471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 626/1235] 10.222.0.70 () {56 vars in 1102 bytes} [Wed Aug 24 16:32:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:05.084322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9732d6f-48ae-4bef-951a-80e0df76a923 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:05.131501 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:05.185935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9732d6f-48ae-4bef-951a-80e0df76a923 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:05.219391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eaed3-d694-48f8-a664-9585e43c4951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:05.220040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eaed3-d694-48f8-a664-9585e43c4951 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:05.220266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eaed3-d694-48f8-a664-9585e43c4951 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:05.220569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eaed3-d694-48f8-a664-9585e43c4951 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:05.221807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483eaed3-d694-48f8-a664-9585e43c4951 None admin] RBAC: Adding query filter params (name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:05.222220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483eaed3-d694-48f8-a664-9585e43c4951 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:05.223584 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483eaed3-d694-48f8-a664-9585e43c4951 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:05.231436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 610/1236] 10.222.0.70 () {56 vars in 1103 bytes} [Wed Aug 24 16:32:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:05.237330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824f35e-06b2-4b27-9d83-82f2ab247678 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:05.321853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:05.337757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824f35e-06b2-4b27-9d83-82f2ab247678 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:05.371514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef91f3d-76e8-487f-961c-d71d3a098306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:05.372153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef91f3d-76e8-487f-961c-d71d3a098306 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:05.372457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef91f3d-76e8-487f-961c-d71d3a098306 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:05.372737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef91f3d-76e8-487f-961c-d71d3a098306 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:05.384323 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef91f3d-76e8-487f-961c-d71d3a098306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:05.385788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef91f3d-76e8-487f-961c-d71d3a098306 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-aef91f3d-76e8-487f-961c-d71d3a098306 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:32:05.400260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:05.401500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:32:05.402787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:32:05.402787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:05.402787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:32:05.402787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 627/1237] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:05.407285 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eaed3-d694-48f8-a664-9585e43c4951 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:05.507236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eaed3-d694-48f8-a664-9585e43c4951 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:05.516066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:05.540694 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224509a1-cf2d-4cee-aad9-9cdb93885a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:05.541809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224509a1-cf2d-4cee-aad9-9cdb93885a7d None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:05.541809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224509a1-cf2d-4cee-aad9-9cdb93885a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:05.542010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224509a1-cf2d-4cee-aad9-9cdb93885a7d None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:05.542474 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224509a1-cf2d-4cee-aad9-9cdb93885a7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:05.542778 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224509a1-cf2d-4cee-aad9-9cdb93885a7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:05.544291 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224509a1-cf2d-4cee-aad9-9cdb93885a7d None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:05.556703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 611/1238] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:05.562782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef91f3d-76e8-487f-961c-d71d3a098306 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:05.664967 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef91f3d-76e8-487f-961c-d71d3a098306 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:05.699249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7822f-139b-49a5-80a6-8f98a7b3f06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:05.699867 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7822f-139b-49a5-80a6-8f98a7b3f06a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:05.700142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7822f-139b-49a5-80a6-8f98a7b3f06a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:05.700427 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7822f-139b-49a5-80a6-8f98a7b3f06a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:05.703046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:05.708075 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7822f-139b-49a5-80a6-8f98a7b3f06a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:05.709540 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7822f-139b-49a5-80a6-8f98a7b3f06a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-89d7822f-139b-49a5-80a6-8f98a7b3f06a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:32:05.720263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:05.721426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:05.722429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:05.722429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:05.722429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:32:05.722429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:32:05.722429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:32:05.722429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:32:05.722429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:32:05.722429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:32:05.722429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:32:05.722429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:32:05.722429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 628/1239] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:32:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:32:05.727656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224509a1-cf2d-4cee-aad9-9cdb93885a7d None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:05.828154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224509a1-cf2d-4cee-aad9-9cdb93885a7d None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:05.862029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96080866-627c-4e51-9c5b-206853d5fdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:05.862712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96080866-627c-4e51-9c5b-206853d5fdbf None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:05.863018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96080866-627c-4e51-9c5b-206853d5fdbf None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:05.863289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96080866-627c-4e51-9c5b-206853d5fdbf None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:05.864004 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96080866-627c-4e51-9c5b-206853d5fdbf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:05.864306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96080866-627c-4e51-9c5b-206853d5fdbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:05.865764 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96080866-627c-4e51-9c5b-206853d5fdbf None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:05.884195 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 612/1240] 10.222.0.70 () {56 vars in 1143 bytes} [Wed Aug 24 16:32:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:05.890556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7822f-139b-49a5-80a6-8f98a7b3f06a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:05.988641 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7822f-139b-49a5-80a6-8f98a7b3f06a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:06.022230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef17ee-ec81-4308-bf92-4c9150dbaa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:06.022838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef17ee-ec81-4308-bf92-4c9150dbaa4d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:06.023120 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef17ee-ec81-4308-bf92-4c9150dbaa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:06.023423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef17ee-ec81-4308-bf92-4c9150dbaa4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:06.084907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:06.091288 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef17ee-ec81-4308-bf92-4c9150dbaa4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:06.092782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef17ee-ec81-4308-bf92-4c9150dbaa4d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:06.188432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 629/1241] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:32:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:32:06.195218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96080866-627c-4e51-9c5b-206853d5fdbf None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:06.223883 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:06.296062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96080866-627c-4e51-9c5b-206853d5fdbf None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:06.331241 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bade47c-496b-4c06-a35d-4a62b1b5b36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:06.331936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bade47c-496b-4c06-a35d-4a62b1b5b36e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:06.332218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bade47c-496b-4c06-a35d-4a62b1b5b36e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:06.332510 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bade47c-496b-4c06-a35d-4a62b1b5b36e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:06.342712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bade47c-496b-4c06-a35d-4a62b1b5b36e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:06.344114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bade47c-496b-4c06-a35d-4a62b1b5b36e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-2bade47c-496b-4c06-a35d-4a62b1b5b36e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:32:06.358380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:06.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:32:06.360716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:32:06.360716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:06.360716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:06.360716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 613/1242] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:06.365531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef17ee-ec81-4308-bf92-4c9150dbaa4d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:06.464597 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:06.470479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef17ee-ec81-4308-bf92-4c9150dbaa4d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:06.503946 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064cc56-8d8a-4baf-9eb3-25b7ac712366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:06.504575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064cc56-8d8a-4baf-9eb3-25b7ac712366 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:06.504851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064cc56-8d8a-4baf-9eb3-25b7ac712366 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:06.505117 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064cc56-8d8a-4baf-9eb3-25b7ac712366 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:06.505746 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064cc56-8d8a-4baf-9eb3-25b7ac712366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:06.506043 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064cc56-8d8a-4baf-9eb3-25b7ac712366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:06.507384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064cc56-8d8a-4baf-9eb3-25b7ac712366 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:06.518364 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 630/1243] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:06.524245 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bade47c-496b-4c06-a35d-4a62b1b5b36e None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:06.602342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:06.623670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bade47c-496b-4c06-a35d-4a62b1b5b36e None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:06.659438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c78bd0-5889-4fb0-a039-2c40e665b56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:06.660059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c78bd0-5889-4fb0-a039-2c40e665b56c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:06.660344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c78bd0-5889-4fb0-a039-2c40e665b56c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:06.660641 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c78bd0-5889-4fb0-a039-2c40e665b56c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:06.667303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c78bd0-5889-4fb0-a039-2c40e665b56c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:06.668862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c78bd0-5889-4fb0-a039-2c40e665b56c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:06.681014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-a7c78bd0-5889-4fb0-a039-2c40e665b56c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:32:06.681014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:06.681014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:06.681014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:06.681014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:06.681014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:06.681014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:06.681014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:06.681014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:06.681014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:06.681014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:06.681014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:06.681014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:32:06.681014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:32:06.681014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:06.683330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:06.684466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:32:06.684466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:32:06.684466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:32:06.684466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:32:06.684466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:32:06.684466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:32:06.684466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:32:06.684466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:06.684466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 614/1244] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:32:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:32:06.688140 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064cc56-8d8a-4baf-9eb3-25b7ac712366 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:06.787457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064cc56-8d8a-4baf-9eb3-25b7ac712366 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:06.821419 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bba75-efcc-4f42-85e4-293ec27a63fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:06.822019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bba75-efcc-4f42-85e4-293ec27a63fe None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:06.822293 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bba75-efcc-4f42-85e4-293ec27a63fe None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:06.822563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bba75-efcc-4f42-85e4-293ec27a63fe None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:06.823239 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44bba75-efcc-4f42-85e4-293ec27a63fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:06.823554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44bba75-efcc-4f42-85e4-293ec27a63fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:06.824983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44bba75-efcc-4f42-85e4-293ec27a63fe None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:06.836718 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 631/1245] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:32:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:32:06.843069 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c78bd0-5889-4fb0-a039-2c40e665b56c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:06.946851 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c78bd0-5889-4fb0-a039-2c40e665b56c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:06.980146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96df91f-6b1c-4944-b837-0bb770eb0183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:06.980775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96df91f-6b1c-4944-b837-0bb770eb0183 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:06.981060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96df91f-6b1c-4944-b837-0bb770eb0183 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:06.981335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96df91f-6b1c-4944-b837-0bb770eb0183 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:06.982064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96df91f-6b1c-4944-b837-0bb770eb0183 None admin] RBAC: Adding query filter params (role.id=bb34a44869834493b10a99cc751ab6c2, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=fc4c43c85a5d49979ba803fdb1041c4e) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:06.982374 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96df91f-6b1c-4944-b837-0bb770eb0183 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:06.983809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96df91f-6b1c-4944-b837-0bb770eb0183 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:06.990237 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 615/1246] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:32:06 2022] GET /identity/v3/role_assignments?user.id=fc4c43c85a5d49979ba803fdb1041c4e&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=bb34a44869834493b10a99cc751ab6c2 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:08.963321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4688e8-2a19-41fa-b2aa-75a884958456 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:08.963862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4688e8-2a19-41fa-b2aa-75a884958456 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:08.963862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4688e8-2a19-41fa-b2aa-75a884958456 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:08.965241 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 632/1247] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:08.970469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d9c4a-6706-4516-b237-36c0ffb93427 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:08.971539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d9c4a-6706-4516-b237-36c0ffb93427 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:08.971678 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d9c4a-6706-4516-b237-36c0ffb93427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:08.978036 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:08.990309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-ce7d9c4a-6706-4516-b237-36c0ffb93427 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:08.991518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ce7d9c4a-6706-4516-b237-36c0ffb93427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:09.057870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 616/1248] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:32:09.327703 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4688e8-2a19-41fa-b2aa-75a884958456 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:09.346406 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:09.428623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4688e8-2a19-41fa-b2aa-75a884958456 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:09.463541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cad76d-2aa1-46cd-ab8b-06ed499fc0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:09.464162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cad76d-2aa1-46cd-ab8b-06ed499fc0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:09.464460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cad76d-2aa1-46cd-ab8b-06ed499fc0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:09.464730 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cad76d-2aa1-46cd-ab8b-06ed499fc0f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:09.474558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cad76d-2aa1-46cd-ab8b-06ed499fc0f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-e0cad76d-2aa1-46cd-ab8b-06ed499fc0f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:32:09.479259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:09.480501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:09.481631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:09.481631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:09.481631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:09.481631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:09.481631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:32:09.481631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:32:09.481631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:32:09.481631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:32:09.481631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:32:09.481631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:32:09.481631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:32:09.481631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:32:09.481631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 633/1249] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:32:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:09.487134 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d9c4a-6706-4516-b237-36c0ffb93427 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:09.588948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d9c4a-6706-4516-b237-36c0ffb93427 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:09.618686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:09.623673 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cabf1f-9198-4c44-ba4a-3c1e5cce843a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:09.624286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cabf1f-9198-4c44-ba4a-3c1e5cce843a None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:09.624599 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cabf1f-9198-4c44-ba4a-3c1e5cce843a None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:09.624871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cabf1f-9198-4c44-ba4a-3c1e5cce843a None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:09.625532 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cabf1f-9198-4c44-ba4a-3c1e5cce843a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:09.625836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cabf1f-9198-4c44-ba4a-3c1e5cce843a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:09.627827 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cabf1f-9198-4c44-ba4a-3c1e5cce843a None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:09.635995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 617/1250] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:32:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:09.642241 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cad76d-2aa1-46cd-ab8b-06ed499fc0f8 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:09.718291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:09.741589 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cad76d-2aa1-46cd-ab8b-06ed499fc0f8 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:09.775466 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a8d51-270d-4ba8-a917-8a111a01a9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:09.776087 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a8d51-270d-4ba8-a917-8a111a01a9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:09.776366 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a8d51-270d-4ba8-a917-8a111a01a9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:09.776687 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a8d51-270d-4ba8-a917-8a111a01a9c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:09.786334 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a8d51-270d-4ba8-a917-8a111a01a9c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:09.787787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a8d51-270d-4ba8-a917-8a111a01a9c9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f83a8d51-270d-4ba8-a917-8a111a01a9c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:32:09.802599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:09.805752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:32:09.806882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:32:09.806882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:09.806882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:32:09.806882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 634/1251] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:32:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:09.809952 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cabf1f-9198-4c44-ba4a-3c1e5cce843a None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:09.911887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cabf1f-9198-4c44-ba4a-3c1e5cce843a None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:09.945715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c24921-eb37-4d23-b903-b7253a346af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:09.946319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c24921-eb37-4d23-b903-b7253a346af5 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:09.946599 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c24921-eb37-4d23-b903-b7253a346af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:09.946869 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c24921-eb37-4d23-b903-b7253a346af5 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:09.947504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c24921-eb37-4d23-b903-b7253a346af5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:09.947800 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c24921-eb37-4d23-b903-b7253a346af5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:09.949180 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c24921-eb37-4d23-b903-b7253a346af5 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:09.960721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 618/1252] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:09.966445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a8d51-270d-4ba8-a917-8a111a01a9c9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:10.066748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a8d51-270d-4ba8-a917-8a111a01a9c9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:10.093209 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:10.101640 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19238844-a5b0-4f0e-ac38-3bd4d2c02af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:10.102271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19238844-a5b0-4f0e-ac38-3bd4d2c02af1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:10.102551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19238844-a5b0-4f0e-ac38-3bd4d2c02af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:10.102823 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19238844-a5b0-4f0e-ac38-3bd4d2c02af1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:10.110373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19238844-a5b0-4f0e-ac38-3bd4d2c02af1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:10.112499 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19238844-a5b0-4f0e-ac38-3bd4d2c02af1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-19238844-a5b0-4f0e-ac38-3bd4d2c02af1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:32:10.123509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:10.124939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:10.126252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:10.126252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:10.126252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:32:10.126252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:32:10.126252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:32:10.126252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:32:10.126252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:32:10.126252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:32:10.126252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:32:10.126252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:32:10.126252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 635/1253] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:32:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:10.130678 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c24921-eb37-4d23-b903-b7253a346af5 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:10.190115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:10.234435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c24921-eb37-4d23-b903-b7253a346af5 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:10.268995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2ceae-6e73-4e35-a5ff-a0ab47195c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:10.269598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2ceae-6e73-4e35-a5ff-a0ab47195c8e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:10.269881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2ceae-6e73-4e35-a5ff-a0ab47195c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:10.270157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2ceae-6e73-4e35-a5ff-a0ab47195c8e None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:10.270882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2ceae-6e73-4e35-a5ff-a0ab47195c8e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:10.271188 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2ceae-6e73-4e35-a5ff-a0ab47195c8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:10.272632 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2ceae-6e73-4e35-a5ff-a0ab47195c8e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:10.290371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 619/1254] 10.222.0.70 () {56 vars in 1143 bytes} [Wed Aug 24 16:32:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:10.296206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19238844-a5b0-4f0e-ac38-3bd4d2c02af1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:10.395898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19238844-a5b0-4f0e-ac38-3bd4d2c02af1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:10.428970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e3975-37d8-42f9-bab4-36ea40a95644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:10.429598 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e3975-37d8-42f9-bab4-36ea40a95644 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:10.429986 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e3975-37d8-42f9-bab4-36ea40a95644 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:10.430256 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e3975-37d8-42f9-bab4-36ea40a95644 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:10.485951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:10.497985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e3975-37d8-42f9-bab4-36ea40a95644 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:10.499415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e3975-37d8-42f9-bab4-36ea40a95644 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:10.598931 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 636/1255] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:32:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:32:10.604970 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2ceae-6e73-4e35-a5ff-a0ab47195c8e None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:10.703542 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2ceae-6e73-4e35-a5ff-a0ab47195c8e None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:10.717052 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:10.737344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117754dc-e4e6-4fa7-a973-34607de53e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:10.737904 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117754dc-e4e6-4fa7-a973-34607de53e86 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:10.738189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117754dc-e4e6-4fa7-a973-34607de53e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:10.738441 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117754dc-e4e6-4fa7-a973-34607de53e86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:10.748698 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117754dc-e4e6-4fa7-a973-34607de53e86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:10.750129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117754dc-e4e6-4fa7-a973-34607de53e86 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-117754dc-e4e6-4fa7-a973-34607de53e86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:32:10.765726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:10.766892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:32:10.768639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:32:10.768639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:10.768639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:10.768639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 620/1256] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:32:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:10.773472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e3975-37d8-42f9-bab4-36ea40a95644 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:10.859399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:10.871369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e3975-37d8-42f9-bab4-36ea40a95644 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:10.905737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b9de9-4bd3-4400-9e7d-921891f98f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:10.906344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b9de9-4bd3-4400-9e7d-921891f98f64 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:10.906631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b9de9-4bd3-4400-9e7d-921891f98f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:10.906901 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b9de9-4bd3-4400-9e7d-921891f98f64 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:10.907545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b9de9-4bd3-4400-9e7d-921891f98f64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:10.907848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b9de9-4bd3-4400-9e7d-921891f98f64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:10.909276 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b9de9-4bd3-4400-9e7d-921891f98f64 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:10.921127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 637/1257] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:10.927598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117754dc-e4e6-4fa7-a973-34607de53e86 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:11.029304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117754dc-e4e6-4fa7-a973-34607de53e86 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:11.063907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b6589-d4a6-4731-9f0d-1abfffc7d539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:11.065118 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b6589-d4a6-4731-9f0d-1abfffc7d539 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:11.065392 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b6589-d4a6-4731-9f0d-1abfffc7d539 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:11.065658 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b6589-d4a6-4731-9f0d-1abfffc7d539 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:11.072556 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b6589-d4a6-4731-9f0d-1abfffc7d539 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:11.073992 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b6589-d4a6-4731-9f0d-1abfffc7d539 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:11.086277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-6a1b6589-d4a6-4731-9f0d-1abfffc7d539 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:32:11.086277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:11.086277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:11.086277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:11.086277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:11.086277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:11.086277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:11.086277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:11.086277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:11.086277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:11.086277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:11.086277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:11.086277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:32:11.086277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:32:11.086277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:11.096064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:11.097303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:32:11.097303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:32:11.097303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:32:11.097303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:32:11.097303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:32:11.097303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:32:11.097303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:32:11.097303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:11.097303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 621/1258] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:32:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:11.097922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b9de9-4bd3-4400-9e7d-921891f98f64 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:11.194725 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b9de9-4bd3-4400-9e7d-921891f98f64 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:11.227629 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b62eff-f142-4ae8-b051-bd4c58529ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:11.228223 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b62eff-f142-4ae8-b051-bd4c58529ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:11.228631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b62eff-f142-4ae8-b051-bd4c58529ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:11.228877 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b62eff-f142-4ae8-b051-bd4c58529ef0 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:11.229478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b62eff-f142-4ae8-b051-bd4c58529ef0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:11.229775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b62eff-f142-4ae8-b051-bd4c58529ef0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:11.231175 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b62eff-f142-4ae8-b051-bd4c58529ef0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:11.243639 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 638/1259] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:32:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:11.249822 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b6589-d4a6-4731-9f0d-1abfffc7d539 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:11.276884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:11.351669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b6589-d4a6-4731-9f0d-1abfffc7d539 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:11.386274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff000b8-a119-49e7-b01d-339472b71656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:11.388006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff000b8-a119-49e7-b01d-339472b71656 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:11.388006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff000b8-a119-49e7-b01d-339472b71656 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:11.388006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff000b8-a119-49e7-b01d-339472b71656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:11.388256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff000b8-a119-49e7-b01d-339472b71656 None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=fc4c43c85a5d49979ba803fdb1041c4e) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:11.388435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff000b8-a119-49e7-b01d-339472b71656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:11.389856 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff000b8-a119-49e7-b01d-339472b71656 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:11.395813 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 622/1260] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:32:11 2022] GET /identity/v3/role_assignments?user.id=fc4c43c85a5d49979ba803fdb1041c4e&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=e7f48343fc1542979ab4459865109e30 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:13.276226 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a59378-130a-4bde-97ce-1fdd1645b15a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:13.276903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a59378-130a-4bde-97ce-1fdd1645b15a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:13.277002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a59378-130a-4bde-97ce-1fdd1645b15a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:13.278330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 639/1261] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:13.283839 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb62b77-6eda-4213-aa92-ffa10a38e735 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:13.284256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb62b77-6eda-4213-aa92-ffa10a38e735 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:13.284370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb62b77-6eda-4213-aa92-ffa10a38e735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:13.304686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-7bb62b77-6eda-4213-aa92-ffa10a38e735 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:13.305877 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7bb62b77-6eda-4213-aa92-ffa10a38e735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:13.375147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 623/1262] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:32:13.646896 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a59378-130a-4bde-97ce-1fdd1645b15a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:13.658792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:13.749794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a59378-130a-4bde-97ce-1fdd1645b15a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:13.783785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a6bbb-f4a0-401e-9c24-048aea9192d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:13.784436 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a6bbb-f4a0-401e-9c24-048aea9192d1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:13.784727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a6bbb-f4a0-401e-9c24-048aea9192d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:13.784999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a6bbb-f4a0-401e-9c24-048aea9192d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:13.795005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a6bbb-f4a0-401e-9c24-048aea9192d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-837a6bbb-f4a0-401e-9c24-048aea9192d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:32:13.800196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:13.801400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:13.802444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:13.802444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:13.802444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:13.802444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:13.802444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:32:13.802444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:32:13.802444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:32:13.802444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:32:13.802444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:32:13.802444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:32:13.802444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:32:13.802444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:32:13.802444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 640/1263] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:32:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:13.807451 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb62b77-6eda-4213-aa92-ffa10a38e735 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:13.826418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:13.917380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb62b77-6eda-4213-aa92-ffa10a38e735 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:13.951702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28986d5a-0772-4c2f-b4ae-924c742c3ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:13.952311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28986d5a-0772-4c2f-b4ae-924c742c3ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:13.952631 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28986d5a-0772-4c2f-b4ae-924c742c3ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:13.952903 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28986d5a-0772-4c2f-b4ae-924c742c3ec3 None admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:13.953560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28986d5a-0772-4c2f-b4ae-924c742c3ec3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:13.953861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28986d5a-0772-4c2f-b4ae-924c742c3ec3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:13.955727 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28986d5a-0772-4c2f-b4ae-924c742c3ec3 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:13.963704 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 624/1264] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:32:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:32:13.969624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a6bbb-f4a0-401e-9c24-048aea9192d1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:14.037709 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:14.070189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a6bbb-f4a0-401e-9c24-048aea9192d1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:14.103682 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f57a6-a654-4687-9732-fbc8bec550d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:14.104306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f57a6-a654-4687-9732-fbc8bec550d9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:14.104620 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f57a6-a654-4687-9732-fbc8bec550d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:14.104892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f57a6-a654-4687-9732-fbc8bec550d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:14.114765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f57a6-a654-4687-9732-fbc8bec550d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:14.116197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f57a6-a654-4687-9732-fbc8bec550d9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-a85f57a6-a654-4687-9732-fbc8bec550d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:32:14.130260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:14.135810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:32:14.137642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:32:14.137642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:14.137642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:32:14.137642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 641/1265] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:14.137934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28986d5a-0772-4c2f-b4ae-924c742c3ec3 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:14.230695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:14.249347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28986d5a-0772-4c2f-b4ae-924c742c3ec3 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:14.282548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465add7-c015-4a79-ba8a-8061e97e5320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:14.283759 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465add7-c015-4a79-ba8a-8061e97e5320 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:14.284039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465add7-c015-4a79-ba8a-8061e97e5320 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:14.284634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465add7-c015-4a79-ba8a-8061e97e5320 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:14.284969 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465add7-c015-4a79-ba8a-8061e97e5320 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:14.285269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465add7-c015-4a79-ba8a-8061e97e5320 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:14.286618 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465add7-c015-4a79-ba8a-8061e97e5320 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:14.298117 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 625/1266] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:14.303947 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f57a6-a654-4687-9732-fbc8bec550d9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:14.403884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f57a6-a654-4687-9732-fbc8bec550d9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:14.423093 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:14.436691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8892bda-a1f3-4716-8ac0-d51adcbb93b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:14.437312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8892bda-a1f3-4716-8ac0-d51adcbb93b3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:14.437587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8892bda-a1f3-4716-8ac0-d51adcbb93b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:14.437858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8892bda-a1f3-4716-8ac0-d51adcbb93b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:14.445801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8892bda-a1f3-4716-8ac0-d51adcbb93b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:14.447259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8892bda-a1f3-4716-8ac0-d51adcbb93b3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-d8892bda-a1f3-4716-8ac0-d51adcbb93b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:32:14.459200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:14.460377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:14.461594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:14.461594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:14.461594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:32:14.461594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:32:14.461594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:32:14.461594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:32:14.461594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:32:14.461594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:32:14.461594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:32:14.461594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:32:14.461594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 642/1267] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:32:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:14.466253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465add7-c015-4a79-ba8a-8061e97e5320 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:14.568161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465add7-c015-4a79-ba8a-8061e97e5320 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:14.602156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba6334-e64d-48f5-b5c2-52c9022ab56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:14.602757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba6334-e64d-48f5-b5c2-52c9022ab56d None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:14.603035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba6334-e64d-48f5-b5c2-52c9022ab56d None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:14.603340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba6334-e64d-48f5-b5c2-52c9022ab56d None admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:14.604024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba6334-e64d-48f5-b5c2-52c9022ab56d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:14.604328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba6334-e64d-48f5-b5c2-52c9022ab56d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:14.605782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba6334-e64d-48f5-b5c2-52c9022ab56d None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:14.614880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:14.624246 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 626/1268] 10.222.0.70 () {56 vars in 1143 bytes} [Wed Aug 24 16:32:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:14.630678 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8892bda-a1f3-4716-8ac0-d51adcbb93b3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:14.733449 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8892bda-a1f3-4716-8ac0-d51adcbb93b3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:14.767856 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96f6ff-f459-4dbb-81ca-844c06c768e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:14.768510 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96f6ff-f459-4dbb-81ca-844c06c768e3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:14.768796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96f6ff-f459-4dbb-81ca-844c06c768e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:14.769072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96f6ff-f459-4dbb-81ca-844c06c768e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:14.779704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96f6ff-f459-4dbb-81ca-844c06c768e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:14.781182 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96f6ff-f459-4dbb-81ca-844c06c768e3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-7d96f6ff-f459-4dbb-81ca-844c06c768e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:32:14.794765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:14.805168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:32:14.806312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:32:14.806312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:14.806312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:32:14.806312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 643/1269] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:32:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:14.806600 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba6334-e64d-48f5-b5c2-52c9022ab56d None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:14.903026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba6334-e64d-48f5-b5c2-52c9022ab56d None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:14.937401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc67b04-1996-4c29-95c6-02348a44bcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:14.938009 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc67b04-1996-4c29-95c6-02348a44bcab None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:14.938293 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc67b04-1996-4c29-95c6-02348a44bcab None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:14.938567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc67b04-1996-4c29-95c6-02348a44bcab None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:14.939751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc67b04-1996-4c29-95c6-02348a44bcab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:14.940055 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc67b04-1996-4c29-95c6-02348a44bcab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:14.941452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc67b04-1996-4c29-95c6-02348a44bcab None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:14.952638 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 627/1270] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:14.958523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96f6ff-f459-4dbb-81ca-844c06c768e3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:14.991025 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:15.057071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96f6ff-f459-4dbb-81ca-844c06c768e3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:15.090696 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21068e-dffd-4cfa-af87-f7a0e7259abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:15.091491 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21068e-dffd-4cfa-af87-f7a0e7259abc None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:15.091612 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21068e-dffd-4cfa-af87-f7a0e7259abc None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:15.091844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21068e-dffd-4cfa-af87-f7a0e7259abc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:15.099084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d21068e-dffd-4cfa-af87-f7a0e7259abc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:15.100542 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d21068e-dffd-4cfa-af87-f7a0e7259abc None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:15.112000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-6d21068e-dffd-4cfa-af87-f7a0e7259abc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:32:15.112000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:15.112000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:15.112000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:15.112000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:15.112000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:15.112000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:15.112000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:15.112000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:15.112000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:15.112000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:15.112000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:15.112000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:32:15.112000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:32:15.112000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:15.113106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:15.114517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:32:15.114517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:32:15.114517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:32:15.114517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:32:15.114517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:32:15.114517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:32:15.114517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:32:15.114517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:32:15.114517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 644/1271] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:32:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:15.119115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc67b04-1996-4c29-95c6-02348a44bcab None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:15.180524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:15.222376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc67b04-1996-4c29-95c6-02348a44bcab None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:15.258187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d3c22-0012-4f5e-be99-bb227db078dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:15.258709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d3c22-0012-4f5e-be99-bb227db078dd None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:15.258986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d3c22-0012-4f5e-be99-bb227db078dd None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:15.259254 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d3c22-0012-4f5e-be99-bb227db078dd None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:15.259934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d3c22-0012-4f5e-be99-bb227db078dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:15.260233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d3c22-0012-4f5e-be99-bb227db078dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:15.261940 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d3c22-0012-4f5e-be99-bb227db078dd None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:15.274055 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 628/1272] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:32:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:15.280061 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21068e-dffd-4cfa-af87-f7a0e7259abc None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:15.372269 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:15.380824 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21068e-dffd-4cfa-af87-f7a0e7259abc None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:15.413628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfcedf-2c6b-47fd-a8ad-5e6185bb7173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:15.414283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfcedf-2c6b-47fd-a8ad-5e6185bb7173 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:15.414565 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfcedf-2c6b-47fd-a8ad-5e6185bb7173 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:15.414838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfcedf-2c6b-47fd-a8ad-5e6185bb7173 None admin] PATH_INFO: `/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/fc4c43c85a5d49979ba803fdb1041c4e/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:15.416746 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfcedf-2c6b-47fd-a8ad-5e6185bb7173 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc920dce30734ac7a69dc039d8c74ce1, user_id=fc4c43c85a5d49979ba803fdb1041c4e, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:15.418172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfcedf-2c6b-47fd-a8ad-5e6185bb7173 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:15.498702 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 645/1273] 10.222.0.70 () {58 vars in 1439 bytes} [Wed Aug 24 16:32:15 2022] PUT /identity/v3/projects/cc920dce30734ac7a69dc039d8c74ce1/users/fc4c43c85a5d49979ba803fdb1041c4e/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:32:17.420289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554500f-d27a-44df-9b19-d7ac56b9a237 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:17.420920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554500f-d27a-44df-9b19-d7ac56b9a237 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:17.420920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554500f-d27a-44df-9b19-d7ac56b9a237 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:17.422219 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 629/1274] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:17.428123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3c400-459e-45d0-9084-1422e323ddc5 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:17.434106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3c400-459e-45d0-9084-1422e323ddc5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:17.434106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3c400-459e-45d0-9084-1422e323ddc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:17.447433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b8f3c400-459e-45d0-9084-1422e323ddc5 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:17.448667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b8f3c400-459e-45d0-9084-1422e323ddc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:17.515899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 646/1275] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:32:17.790999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554500f-d27a-44df-9b19-d7ac56b9a237 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:17.896137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554500f-d27a-44df-9b19-d7ac56b9a237 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:17.916360 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:17.933233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465547d9-36f1-4ff5-b126-a4f3478efab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:17.933861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465547d9-36f1-4ff5-b126-a4f3478efab0 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:17.934145 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465547d9-36f1-4ff5-b126-a4f3478efab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:17.934419 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465547d9-36f1-4ff5-b126-a4f3478efab0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:17.944479 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465547d9-36f1-4ff5-b126-a4f3478efab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-465547d9-36f1-4ff5-b126-a4f3478efab0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:32:17.949163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:17.950296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:17.951153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:17.951153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:17.951153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:17.951153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:17.951153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:32:17.951153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:32:17.951153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:32:17.951153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:32:17.951153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:32:17.951153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:32:17.951153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:32:17.951153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:17.951153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 630/1276] 10.222.0.70 () {56 vars in 1096 bytes} [Wed Aug 24 16:32:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:17.956958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3c400-459e-45d0-9084-1422e323ddc5 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:18.037951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:18.055284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3c400-459e-45d0-9084-1422e323ddc5 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:18.089162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74594053-9f64-4242-892b-76e2d7e2a84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:18.089768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74594053-9f64-4242-892b-76e2d7e2a84b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:18.090044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74594053-9f64-4242-892b-76e2d7e2a84b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:18.090317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74594053-9f64-4242-892b-76e2d7e2a84b None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:18.090977 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74594053-9f64-4242-892b-76e2d7e2a84b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:18.091275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74594053-9f64-4242-892b-76e2d7e2a84b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:18.092692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74594053-9f64-4242-892b-76e2d7e2a84b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:18.100981 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 647/1277] 10.222.0.70 () {56 vars in 1099 bytes} [Wed Aug 24 16:32:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:18.106740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465547d9-36f1-4ff5-b126-a4f3478efab0 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:18.206350 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465547d9-36f1-4ff5-b126-a4f3478efab0 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:18.239905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6361d5-ec6b-4ce6-9e7a-965e8bf81d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:18.240552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6361d5-ec6b-4ce6-9e7a-965e8bf81d2c None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:18.240834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6361d5-ec6b-4ce6-9e7a-965e8bf81d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:18.241103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6361d5-ec6b-4ce6-9e7a-965e8bf81d2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:18.250805 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6361d5-ec6b-4ce6-9e7a-965e8bf81d2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:18.252768 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6361d5-ec6b-4ce6-9e7a-965e8bf81d2c None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-2d6361d5-ec6b-4ce6-9e7a-965e8bf81d2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:32:18.267576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:18.268725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:32:18.274399 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:32:18.274399 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:18.274399 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:18.274399 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 631/1278] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:32:18.276084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74594053-9f64-4242-892b-76e2d7e2a84b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:18.372719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74594053-9f64-4242-892b-76e2d7e2a84b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:18.406175 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e8583-8a32-4767-abc0-77bb1751b0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:18.406773 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e8583-8a32-4767-abc0-77bb1751b0ae None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:18.407048 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e8583-8a32-4767-abc0-77bb1751b0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:18.407316 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e8583-8a32-4767-abc0-77bb1751b0ae None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:18.407978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e8583-8a32-4767-abc0-77bb1751b0ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:18.408280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e8583-8a32-4767-abc0-77bb1751b0ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:18.409657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e8583-8a32-4767-abc0-77bb1751b0ae None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:18.423019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 648/1279] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:18.429470 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6361d5-ec6b-4ce6-9e7a-965e8bf81d2c None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:18.462003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:18.531470 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6361d5-ec6b-4ce6-9e7a-965e8bf81d2c None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:18.567528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba22cf6-dca2-4c4a-86f8-6be6829fc772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:18.568154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba22cf6-dca2-4c4a-86f8-6be6829fc772 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:18.568457 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba22cf6-dca2-4c4a-86f8-6be6829fc772 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:18.568730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba22cf6-dca2-4c4a-86f8-6be6829fc772 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:18.576138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba22cf6-dca2-4c4a-86f8-6be6829fc772 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:18.577726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba22cf6-dca2-4c4a-86f8-6be6829fc772 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-3ba22cf6-dca2-4c4a-86f8-6be6829fc772 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:32:18.589372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:18.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:18.597141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:18.597141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:18.597141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:32:18.597141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:32:18.597141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:32:18.597141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:32:18.597141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:32:18.597141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:32:18.597141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:32:18.597141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:18.597141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 632/1280] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:32:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:18.597892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e8583-8a32-4767-abc0-77bb1751b0ae None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:18.601472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:18.702302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e8583-8a32-4767-abc0-77bb1751b0ae None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:18.736188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80c9f6-b395-4682-83a4-2b7db5f0fa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:18.738071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80c9f6-b395-4682-83a4-2b7db5f0fa9e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:18.738071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80c9f6-b395-4682-83a4-2b7db5f0fa9e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:18.738071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80c9f6-b395-4682-83a4-2b7db5f0fa9e None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:18.738071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80c9f6-b395-4682-83a4-2b7db5f0fa9e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:18.738400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80c9f6-b395-4682-83a4-2b7db5f0fa9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:18.739782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80c9f6-b395-4682-83a4-2b7db5f0fa9e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:18.756632 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 649/1281] 10.222.0.70 () {56 vars in 1143 bytes} [Wed Aug 24 16:32:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:18.762752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba22cf6-dca2-4c4a-86f8-6be6829fc772 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:18.855738 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:18.865409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba22cf6-dca2-4c4a-86f8-6be6829fc772 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:18.902352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c0e58-3219-449c-8836-74b84bde1505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:18.902999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c0e58-3219-449c-8836-74b84bde1505 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:18.903277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c0e58-3219-449c-8836-74b84bde1505 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:18.903544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c0e58-3219-449c-8836-74b84bde1505 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:18.975549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c0e58-3219-449c-8836-74b84bde1505 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:18.976955 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c0e58-3219-449c-8836-74b84bde1505 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:19.066407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:19.076874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 633/1282] 10.222.0.70 () {58 vars in 1282 bytes} [Wed Aug 24 16:32:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:32:19.083215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80c9f6-b395-4682-83a4-2b7db5f0fa9e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:19.145295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:19.182926 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80c9f6-b395-4682-83a4-2b7db5f0fa9e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:19.217555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ca99f-9128-48af-b314-b13da510ead0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:19.218182 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ca99f-9128-48af-b314-b13da510ead0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:19.218462 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ca99f-9128-48af-b314-b13da510ead0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:19.218735 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ca99f-9128-48af-b314-b13da510ead0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:19.229219 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ca99f-9128-48af-b314-b13da510ead0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:19.230589 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ca99f-9128-48af-b314-b13da510ead0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-048ca99f-9128-48af-b314-b13da510ead0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:32:19.244890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:19.254313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:32:19.255938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:32:19.255938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:32:19.255938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:32:19.255938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 650/1283] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:32:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:19.256318 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c0e58-3219-449c-8836-74b84bde1505 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:19.354293 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c0e58-3219-449c-8836-74b84bde1505 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:19.388970 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca1736-d87f-4222-a4f1-7da24df84192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:19.389578 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca1736-d87f-4222-a4f1-7da24df84192 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:19.389864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca1736-d87f-4222-a4f1-7da24df84192 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:19.390142 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca1736-d87f-4222-a4f1-7da24df84192 None admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:19.390785 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca1736-d87f-4222-a4f1-7da24df84192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:19.391093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca1736-d87f-4222-a4f1-7da24df84192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:19.392986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca1736-d87f-4222-a4f1-7da24df84192 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:19.404237 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 634/1284] 10.222.0.70 () {56 vars in 1109 bytes} [Wed Aug 24 16:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:19.410574 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ca99f-9128-48af-b314-b13da510ead0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:19.514470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ca99f-9128-48af-b314-b13da510ead0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:19.529499 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:19.553200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6037e-cb8c-421b-8f3f-d7b139c37431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:19.553945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6037e-cb8c-421b-8f3f-d7b139c37431 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:19.554302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6037e-cb8c-421b-8f3f-d7b139c37431 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:19.554652 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6037e-cb8c-421b-8f3f-d7b139c37431 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:19.562222 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6037e-cb8c-421b-8f3f-d7b139c37431 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:19.563677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6037e-cb8c-421b-8f3f-d7b139c37431 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:19.576171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-92e6037e-cb8c-421b-8f3f-d7b139c37431 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:32:19.576171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:19.576171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:19.576171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:19.576171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:19.576171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:19.576171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:19.576171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:19.576171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:19.576171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:19.576171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:19.576171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:19.576171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:32:19.576171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:32:19.576171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:19.577883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:19.580389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:32:19.580389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:32:19.580389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:32:19.580389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:32:19.580389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:32:19.580389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:32:19.580389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:32:19.580389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Aug 24 16:32:19.581733 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 651/1285] 10.222.0.70 () {56 vars in 1111 bytes} [Wed Aug 24 16:32:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:19.585512 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca1736-d87f-4222-a4f1-7da24df84192 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:19.641653 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:19.693713 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca1736-d87f-4222-a4f1-7da24df84192 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:19.728645 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be834e91-7aca-4135-983f-8ce68f74570d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:19.729250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be834e91-7aca-4135-983f-8ce68f74570d None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:19.729527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be834e91-7aca-4135-983f-8ce68f74570d None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:19.729803 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be834e91-7aca-4135-983f-8ce68f74570d None admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:19.730484 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be834e91-7aca-4135-983f-8ce68f74570d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:19.730814 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be834e91-7aca-4135-983f-8ce68f74570d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:19.732248 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be834e91-7aca-4135-983f-8ce68f74570d None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:19.746181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 635/1286] 10.222.0.70 () {56 vars in 1148 bytes} [Wed Aug 24 16:32:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:19.753044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6037e-cb8c-421b-8f3f-d7b139c37431 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:19.851991 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6037e-cb8c-421b-8f3f-d7b139c37431 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:19.887826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e577a2d-63d7-4e76-ad37-ba230b1b4aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:19.888441 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e577a2d-63d7-4e76-ad37-ba230b1b4aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:19.888753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e577a2d-63d7-4e76-ad37-ba230b1b4aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:19.889026 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e577a2d-63d7-4e76-ad37-ba230b1b4aa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:19.889745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e577a2d-63d7-4e76-ad37-ba230b1b4aa1 None admin] RBAC: Adding query filter params (role.id=e7f48343fc1542979ab4459865109e30, scope.project.id=cc920dce30734ac7a69dc039d8c74ce1, user.id=fc4c43c85a5d49979ba803fdb1041c4e) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:19.890053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e577a2d-63d7-4e76-ad37-ba230b1b4aa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:19.891478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e577a2d-63d7-4e76-ad37-ba230b1b4aa1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:19.897951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 652/1287] 10.222.0.70 () {56 vars in 1374 bytes} [Wed Aug 24 16:32:19 2022] GET /identity/v3/role_assignments?user.id=fc4c43c85a5d49979ba803fdb1041c4e&scope.project.id=cc920dce30734ac7a69dc039d8c74ce1&role.id=e7f48343fc1542979ab4459865109e30 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:21.849119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371f3da-dec6-41d3-aa34-374ce210d518 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:21.849662 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371f3da-dec6-41d3-aa34-374ce210d518 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:21.849662 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371f3da-dec6-41d3-aa34-374ce210d518 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:21.850940 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 636/1288] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:21.856335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99137bfd-b4af-4369-a9d3-f42b70cd0699 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:21.863649 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99137bfd-b4af-4369-a9d3-f42b70cd0699 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:21.863649 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99137bfd-b4af-4369-a9d3-f42b70cd0699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:21.876459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-99137bfd-b4af-4369-a9d3-f42b70cd0699 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:21.877670 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-99137bfd-b4af-4369-a9d3-f42b70cd0699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:21.910337 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:21.950176 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 653/1289] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:32:22.223795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371f3da-dec6-41d3-aa34-374ce210d518 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:22.338699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371f3da-dec6-41d3-aa34-374ce210d518 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:22.352208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:22.377803 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f9e92-2a08-459f-b161-abcc69bcf5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:22.381481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f9e92-2a08-459f-b161-abcc69bcf5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:22.381481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f9e92-2a08-459f-b161-abcc69bcf5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:22.381481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f9e92-2a08-459f-b161-abcc69bcf5a6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:22.381481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f9e92-2a08-459f-b161-abcc69bcf5a6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:22.381481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f9e92-2a08-459f-b161-abcc69bcf5a6 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-055f9e92-2a08-459f-b161-abcc69bcf5a6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:22.390002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:22.391574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:22.393274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:22.393274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:22.393274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:22.393274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:22.393274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:22.393274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:22.393274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:22.393274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:22.393274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:22.393274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:22.393274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:32:22.393274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:32:22.393274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:32:22.393274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:22.393274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 637/1290] 10.222.0.70 () {56 vars in 1117 bytes} [Wed Aug 24 16:32:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:22.398016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99137bfd-b4af-4369-a9d3-f42b70cd0699 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:22.501939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99137bfd-b4af-4369-a9d3-f42b70cd0699 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:22.537930 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbddbf76-320e-4db6-a937-b9a5a914b7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:22.544711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddbf76-320e-4db6-a937-b9a5a914b7f7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:22.544711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddbf76-320e-4db6-a937-b9a5a914b7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:22.544711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddbf76-320e-4db6-a937-b9a5a914b7f7 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:22.544711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddbf76-320e-4db6-a937-b9a5a914b7f7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:22.544711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddbf76-320e-4db6-a937-b9a5a914b7f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:22.544711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddbf76-320e-4db6-a937-b9a5a914b7f7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:22.547022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 654/1291] 10.222.0.70 () {56 vars in 1116 bytes} [Wed Aug 24 16:32:22 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:22.553220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f9e92-2a08-459f-b161-abcc69bcf5a6 None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:22.672148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f9e92-2a08-459f-b161-abcc69bcf5a6 None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:22.710016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6fdbc-d937-4ff8-912c-17e30edfad32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:22.710622 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6fdbc-d937-4ff8-912c-17e30edfad32 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:22.710900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6fdbc-d937-4ff8-912c-17e30edfad32 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:22.711172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6fdbc-d937-4ff8-912c-17e30edfad32 None admin] PATH_INFO: `/v3/services` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:22.711806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6fdbc-d937-4ff8-912c-17e30edfad32 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:22.712178 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6fdbc-d937-4ff8-912c-17e30edfad32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:22.713590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6fdbc-d937-4ff8-912c-17e30edfad32 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:22.718644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 638/1292] 10.222.0.70 () {56 vars in 1116 bytes} [Wed Aug 24 16:32:22 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:24.707006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f70582-5be1-49b7-ad83-25962180bd3e None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:24.707614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f70582-5be1-49b7-ad83-25962180bd3e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:24.707614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f70582-5be1-49b7-ad83-25962180bd3e None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:24.709432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 655/1293] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:24.715137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1548c-8e2f-46a1-af19-6631b6d5b912 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:24.715792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1548c-8e2f-46a1-af19-6631b6d5b912 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:24.715792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1548c-8e2f-46a1-af19-6631b6d5b912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:24.740186 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-2da1548c-8e2f-46a1-af19-6631b6d5b912 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:24.741325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2da1548c-8e2f-46a1-af19-6631b6d5b912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:24.780234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:24.821188 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 639/1294] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:32:25.090775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f70582-5be1-49b7-ad83-25962180bd3e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:25.122972 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:25.198952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f70582-5be1-49b7-ad83-25962180bd3e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:25.244956 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0528b-34e2-4188-82de-093a403367d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:25.245700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0528b-34e2-4188-82de-093a403367d4 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:25.246004 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0528b-34e2-4188-82de-093a403367d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:25.246282 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0528b-34e2-4188-82de-093a403367d4 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:25.247072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0528b-34e2-4188-82de-093a403367d4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:25.248648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0528b-34e2-4188-82de-093a403367d4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:25.363069 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 656/1295] 10.222.0.70 () {60 vars in 1141 bytes} [Wed Aug 24 16:32:25 2022] POST /identity/v3/services => generated 246 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:32:27.352762 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e59386-9c2b-4705-9883-a6948e3582e2 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:27.353384 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e59386-9c2b-4705-9883-a6948e3582e2 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:27.353384 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e59386-9c2b-4705-9883-a6948e3582e2 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:27.354836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 640/1296] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:27.360794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc8ed9-e10a-4427-831e-08873d81eed4 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:27.361351 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc8ed9-e10a-4427-831e-08873d81eed4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:27.361351 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc8ed9-e10a-4427-831e-08873d81eed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:27.380754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a9dc8ed9-e10a-4427-831e-08873d81eed4 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:27.382144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a9dc8ed9-e10a-4427-831e-08873d81eed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:27.457538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 657/1297] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:27 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:32:27.728505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e59386-9c2b-4705-9883-a6948e3582e2 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:27.838017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e59386-9c2b-4705-9883-a6948e3582e2 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:27.875715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c26f9-f4f4-4689-9b0f-c35f860b820e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:27.876393 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c26f9-f4f4-4689-9b0f-c35f860b820e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:27.876768 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c26f9-f4f4-4689-9b0f-c35f860b820e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:27.877022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c26f9-f4f4-4689-9b0f-c35f860b820e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:27.877615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c26f9-f4f4-4689-9b0f-c35f860b820e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:27.878998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c26f9-f4f4-4689-9b0f-c35f860b820e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-212c26f9-f4f4-4689-9b0f-c35f860b820e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:27.888691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:27.890062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:27.891042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:27.891042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:27.891042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:27.891042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:27.891042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:27.891042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:27.891042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:27.891042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:27.891042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:27.891042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:27.891042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:32:27.891042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:32:27.891042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:32:27.891042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:27.891042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 641/1298] 10.222.0.70 () {56 vars in 1117 bytes} [Wed Aug 24 16:32:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:27.898563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc8ed9-e10a-4427-831e-08873d81eed4 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:27.919464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:28.005366 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc8ed9-e10a-4427-831e-08873d81eed4 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:28.042867 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673a4b1-f0f4-495e-820c-ba9399584a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:28.043493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673a4b1-f0f4-495e-820c-ba9399584a40 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:28.051994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673a4b1-f0f4-495e-820c-ba9399584a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:28.051994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673a4b1-f0f4-495e-820c-ba9399584a40 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:28.051994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673a4b1-f0f4-495e-820c-ba9399584a40 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:28.051994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673a4b1-f0f4-495e-820c-ba9399584a40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:28.051994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673a4b1-f0f4-495e-820c-ba9399584a40 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:28.052789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 658/1299] 10.222.0.70 () {56 vars in 1116 bytes} [Wed Aug 24 16:32:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:28.059173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c26f9-f4f4-4689-9b0f-c35f860b820e None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:28.096933 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:28.170666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c26f9-f4f4-4689-9b0f-c35f860b820e None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:28.208853 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cfcbc-0146-4688-857a-8cdc3d9d5b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:28.209469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cfcbc-0146-4688-857a-8cdc3d9d5b77 None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:28.209755 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cfcbc-0146-4688-857a-8cdc3d9d5b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:28.210034 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cfcbc-0146-4688-857a-8cdc3d9d5b77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:28.210750 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cfcbc-0146-4688-857a-8cdc3d9d5b77 None admin] RBAC: Adding query filter params (interface=public, service_id=afbc854d8f0343ef9c0025eb3ffd8c52, region_id=RegionOne) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:28.211064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cfcbc-0146-4688-857a-8cdc3d9d5b77 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:28.212464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cfcbc-0146-4688-857a-8cdc3d9d5b77 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:28.218217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 642/1300] 10.222.0.70 () {56 vars in 1251 bytes} [Wed Aug 24 16:32:28 2022] GET /identity/v3/endpoints?service_id=afbc854d8f0343ef9c0025eb3ffd8c52&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:28.226177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673a4b1-f0f4-495e-820c-ba9399584a40 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:28.319051 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:28.338006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673a4b1-f0f4-495e-820c-ba9399584a40 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:28.372697 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57738a5-fadb-42be-8808-6ea1059369d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:28.373342 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57738a5-fadb-42be-8808-6ea1059369d9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:28.373574 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57738a5-fadb-42be-8808-6ea1059369d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:28.373841 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57738a5-fadb-42be-8808-6ea1059369d9 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:28.374441 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57738a5-fadb-42be-8808-6ea1059369d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:28.375799 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57738a5-fadb-42be-8808-6ea1059369d9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:28.382628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 659/1301] 10.222.0.70 () {56 vars in 1087 bytes} [Wed Aug 24 16:32:28 2022] GET /identity/v3/services => generated 2191 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:32:30.397831 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ce737-c136-4bfe-a482-d1b29b4563de None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:30.398359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ce737-c136-4bfe-a482-d1b29b4563de None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:30.398359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ce737-c136-4bfe-a482-d1b29b4563de None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:30.399740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 643/1302] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:30.405606 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bc78c-e6ab-4e97-9559-3df8afc300f1 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:30.406132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bc78c-e6ab-4e97-9559-3df8afc300f1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:30.406230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bc78c-e6ab-4e97-9559-3df8afc300f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:30.426878 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7d4bc78c-e6ab-4e97-9559-3df8afc300f1 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:30.428114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7d4bc78c-e6ab-4e97-9559-3df8afc300f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:30.500983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 660/1303] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:30 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:32:30.770318 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ce737-c136-4bfe-a482-d1b29b4563de None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:30.870969 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:30.878173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ce737-c136-4bfe-a482-d1b29b4563de None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:30.915515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf54ddf-19bc-482f-a672-23ff119dff8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:30.916140 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf54ddf-19bc-482f-a672-23ff119dff8e None admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:30.916493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf54ddf-19bc-482f-a672-23ff119dff8e None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:30.916764 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf54ddf-19bc-482f-a672-23ff119dff8e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:30.917341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf54ddf-19bc-482f-a672-23ff119dff8e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:30.918687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf54ddf-19bc-482f-a672-23ff119dff8e None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application [None req-9bf54ddf-19bc-482f-a672-23ff119dff8e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:32:30.927799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:32:30.929111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:32:30.930173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:32:30.930173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application with Lock( Aug 24 16:32:30.930173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:32:30.930173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:32:30.930173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:32:30.930173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:32:30.930173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:32:30.930173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:32:30.930173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:32:30.930173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:32:30.930173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:32:30.930173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:32:30.930173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:32:30.930173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: ERROR keystone.server.flask.application Aug 24 16:32:30.930173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 644/1304] 10.222.0.70 () {56 vars in 1117 bytes} [Wed Aug 24 16:32:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:32:30.935161 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bc78c-e6ab-4e97-9559-3df8afc300f1 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:31.020130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:31.043620 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bc78c-e6ab-4e97-9559-3df8afc300f1 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:31.085198 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac85e3f-3d02-4c28-83cc-ff50bb0690e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:31.085852 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac85e3f-3d02-4c28-83cc-ff50bb0690e6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:31.086137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac85e3f-3d02-4c28-83cc-ff50bb0690e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:31.086411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac85e3f-3d02-4c28-83cc-ff50bb0690e6 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:31.087066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac85e3f-3d02-4c28-83cc-ff50bb0690e6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:32:31.087372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac85e3f-3d02-4c28-83cc-ff50bb0690e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:31.088853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac85e3f-3d02-4c28-83cc-ff50bb0690e6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:31.094358 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 661/1305] 10.222.0.70 () {56 vars in 1116 bytes} [Wed Aug 24 16:32:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:32:31.100669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf54ddf-19bc-482f-a672-23ff119dff8e None admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:31.208885 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf54ddf-19bc-482f-a672-23ff119dff8e None admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:31.243949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e296ed-387f-4110-9ffd-a19a2b7f0ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:31.244749 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e296ed-387f-4110-9ffd-a19a2b7f0ba0 None admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:31.245121 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e296ed-387f-4110-9ffd-a19a2b7f0ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:31.245479 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e296ed-387f-4110-9ffd-a19a2b7f0ba0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:31.246218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e296ed-387f-4110-9ffd-a19a2b7f0ba0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:31.247620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e296ed-387f-4110-9ffd-a19a2b7f0ba0 None admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:31.327437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 645/1306] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:32:31 2022] POST /identity/v3/endpoints => generated 331 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:32:41.091187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=30 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:41.322657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:43.932072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02390985-9788-4e39-b51b-762652ee87af None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:43.933125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02390985-9788-4e39-b51b-762652ee87af None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:43.933616 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02390985-9788-4e39-b51b-762652ee87af None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:43.940720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139af616-680a-43e9-91ed-61631ee40513 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:43.941606 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139af616-680a-43e9-91ed-61631ee40513 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:43.942111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139af616-680a-43e9-91ed-61631ee40513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:43.945255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 662/1307] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:39:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:44.085983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-139af616-680a-43e9-91ed-61631ee40513 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:44.087482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-139af616-680a-43e9-91ed-61631ee40513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:44.292538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 646/1308] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:39:44.590727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0548e-6105-4fe1-a9f5-8e2417805873 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:44.591113 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0548e-6105-4fe1-a9f5-8e2417805873 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:44.591454 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0548e-6105-4fe1-a9f5-8e2417805873 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:44.593153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 663/1309] 10.222.0.70 () {54 vars in 965 bytes} [Wed Aug 24 16:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:44.599976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92961055-837f-4cbd-b0b6-144680912f21 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:44.600269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92961055-837f-4cbd-b0b6-144680912f21 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:44.601195 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92961055-837f-4cbd-b0b6-144680912f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:44.692288 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-92961055-837f-4cbd-b0b6-144680912f21 None None] MFA Rules not processed for user `942b54c39ae5403c8ae3bcca1c31bf1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:44.693578 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-92961055-837f-4cbd-b0b6-144680912f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:44.717735 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:44.783296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 647/1310] 10.222.0.70 () {58 vars in 1062 bytes} [Wed Aug 24 16:39:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:44.792384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0548e-6105-4fe1-a9f5-8e2417805873 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:44.794837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dee0548e-6105-4fe1-a9f5-8e2417805873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:44.918500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0548e-6105-4fe1-a9f5-8e2417805873 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:45.002007 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcecd9-49a7-4361-9214-71eba4f8eb92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:45.002988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcecd9-49a7-4361-9214-71eba4f8eb92 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:45.002988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcecd9-49a7-4361-9214-71eba4f8eb92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:45.003158 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcecd9-49a7-4361-9214-71eba4f8eb92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:45.004704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a1dcecd9-49a7-4361-9214-71eba4f8eb92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:45.070627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:45.133851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dcecd9-49a7-4361-9214-71eba4f8eb92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:45.135300 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dcecd9-49a7-4361-9214-71eba4f8eb92 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:45.159644 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 664/1311] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:47.401274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49084ba8-a8bc-4f01-8f5e-2c3eba2d9102 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:47.401800 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49084ba8-a8bc-4f01-8f5e-2c3eba2d9102 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:47.401800 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49084ba8-a8bc-4f01-8f5e-2c3eba2d9102 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:47.403856 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 648/1312] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:39:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:47.409262 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f6a0d-4261-4399-abd1-897ab6f810e9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:47.409655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f6a0d-4261-4399-abd1-897ab6f810e9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:47.409820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f6a0d-4261-4399-abd1-897ab6f810e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:47.512588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-cd4f6a0d-4261-4399-abd1-897ab6f810e9 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:47.512588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-cd4f6a0d-4261-4399-abd1-897ab6f810e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:47.616733 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 665/1313] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:47.900021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49084ba8-a8bc-4f01-8f5e-2c3eba2d9102 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:47.901532 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-49084ba8-a8bc-4f01-8f5e-2c3eba2d9102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:48.006207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49084ba8-a8bc-4f01-8f5e-2c3eba2d9102 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:48.086782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:48.089621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159c32b-ffd5-4a90-863f-2b1e03decda5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:48.090273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159c32b-ffd5-4a90-863f-2b1e03decda5 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:48.091314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159c32b-ffd5-4a90-863f-2b1e03decda5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:48.091480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159c32b-ffd5-4a90-863f-2b1e03decda5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:48.093151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7159c32b-ffd5-4a90-863f-2b1e03decda5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:48.205881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7159c32b-ffd5-4a90-863f-2b1e03decda5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:48.207260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7159c32b-ffd5-4a90-863f-2b1e03decda5 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:48.225369 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 649/1314] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:50.630988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15022ed7-4c67-403f-9e88-ae351be9d461 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:50.631522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15022ed7-4c67-403f-9e88-ae351be9d461 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:50.631522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15022ed7-4c67-403f-9e88-ae351be9d461 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:50.633175 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 666/1315] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:39:50.639162 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14995e85-0ec2-42b0-ae5d-7ae3531da17f None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:50.639558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14995e85-0ec2-42b0-ae5d-7ae3531da17f None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:50.639706 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14995e85-0ec2-42b0-ae5d-7ae3531da17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:50.704844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-14995e85-0ec2-42b0-ae5d-7ae3531da17f None None] MFA Rules not processed for user `e0ff5ce14df94fc4a1f5f037539e037c`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:50.706042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-14995e85-0ec2-42b0-ae5d-7ae3531da17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:50.756382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:50.801771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 650/1316] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:39:50 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:51.085731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15022ed7-4c67-403f-9e88-ae351be9d461 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.087371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-15022ed7-4c67-403f-9e88-ae351be9d461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:51.097099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:51.192650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15022ed7-4c67-403f-9e88-ae351be9d461 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:51.266003 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0480d-c665-43a0-b24a-59d5aa1a3260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:51.266607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0480d-c665-43a0-b24a-59d5aa1a3260 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.266886 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0480d-c665-43a0-b24a-59d5aa1a3260 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.267159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0480d-c665-43a0-b24a-59d5aa1a3260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.268813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9fa0480d-c665-43a0-b24a-59d5aa1a3260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:51.330064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:51.378072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0480d-c665-43a0-b24a-59d5aa1a3260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:51.379509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0480d-c665-43a0-b24a-59d5aa1a3260 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:51.397397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 667/1317] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:51 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:54.174842 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbbe8f-fb4c-4a0a-a824-8fa20edca1f1 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.175400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbbe8f-fb4c-4a0a-a824-8fa20edca1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.175400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbbe8f-fb4c-4a0a-a824-8fa20edca1f1 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.176778 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 651/1318] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:54.182125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a3cbf-a0e0-4610-92aa-bdd24b0b6b0e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.182516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a3cbf-a0e0-4610-92aa-bdd24b0b6b0e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.182670 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a3cbf-a0e0-4610-92aa-bdd24b0b6b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.249544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0b5a3cbf-a0e0-4610-92aa-bdd24b0b6b0e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:54.250775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0b5a3cbf-a0e0-4610-92aa-bdd24b0b6b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:54.356136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 668/1319] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:39:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:54.651545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbbe8f-fb4c-4a0a-a824-8fa20edca1f1 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:54.653155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2edbbe8f-fb4c-4a0a-a824-8fa20edca1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:54.759121 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbbe8f-fb4c-4a0a-a824-8fa20edca1f1 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:54.831093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb9217-5287-435b-a04f-6e1434557d7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:54.831621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb9217-5287-435b-a04f-6e1434557d7b service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.831960 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb9217-5287-435b-a04f-6e1434557d7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.832177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb9217-5287-435b-a04f-6e1434557d7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.833820 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-5dfb9217-5287-435b-a04f-6e1434557d7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:54.842392 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:54.945656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb9217-5287-435b-a04f-6e1434557d7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:54.947095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb9217-5287-435b-a04f-6e1434557d7b service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:54.965926 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 652/1320] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:57.994407 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d623f83-279f-4bdc-820f-ac62475300b4 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:57.994968 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d623f83-279f-4bdc-820f-ac62475300b4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:57.995065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d623f83-279f-4bdc-820f-ac62475300b4 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:57.996474 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 669/1321] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:58.001962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0298a-3009-4e2e-87fc-c369668ce750 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:58.002432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0298a-3009-4e2e-87fc-c369668ce750 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:58.002528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0298a-3009-4e2e-87fc-c369668ce750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:58.070901 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-e1b0298a-3009-4e2e-87fc-c369668ce750 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:58.072172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e1b0298a-3009-4e2e-87fc-c369668ce750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:58.135545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:58.177301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 653/1322] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:58.481022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506973e-8ff8-4cf2-843a-b124adc58f99 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:58.481311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506973e-8ff8-4cf2-843a-b124adc58f99 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:58.481580 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506973e-8ff8-4cf2-843a-b124adc58f99 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:58.482957 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 670/1323] 10.222.0.70 () {54 vars in 965 bytes} [Wed Aug 24 16:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:58.489776 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad68cd-9996-4dc9-bbb1-0998a6000ad2 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:58.490108 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad68cd-9996-4dc9-bbb1-0998a6000ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:58.490423 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad68cd-9996-4dc9-bbb1-0998a6000ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:58.588767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-dfad68cd-9996-4dc9-bbb1-0998a6000ad2 None None] MFA Rules not processed for user `942b54c39ae5403c8ae3bcca1c31bf1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:58.590040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-dfad68cd-9996-4dc9-bbb1-0998a6000ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:58.701330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 654/1324] 10.222.0.70 () {58 vars in 1062 bytes} [Wed Aug 24 16:39:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:58.710441 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506973e-8ff8-4cf2-843a-b124adc58f99 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:58.712098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8506973e-8ff8-4cf2-843a-b124adc58f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:58.733261 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:58.820346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506973e-8ff8-4cf2-843a-b124adc58f99 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:58.913857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478175f-e083-4bb6-8551-b810103eb563 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:58.914484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478175f-e083-4bb6-8551-b810103eb563 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:58.914765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478175f-e083-4bb6-8551-b810103eb563 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:58.915041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478175f-e083-4bb6-8551-b810103eb563 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:58.916699 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0478175f-e083-4bb6-8551-b810103eb563 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:58.994856 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:59.034757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478175f-e083-4bb6-8551-b810103eb563 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:59.036198 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478175f-e083-4bb6-8551-b810103eb563 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:59.055306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 671/1325] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:01.298897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c1835-eb01-47dc-8c2a-4ae38c949b8d None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.299435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c1835-eb01-47dc-8c2a-4ae38c949b8d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.299435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c1835-eb01-47dc-8c2a-4ae38c949b8d None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.300822 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 655/1326] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:01.306302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77f03d-75d0-477e-a47e-05eecad8cd97 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.306729 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77f03d-75d0-477e-a47e-05eecad8cd97 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.306849 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77f03d-75d0-477e-a47e-05eecad8cd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.392989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fb77f03d-75d0-477e-a47e-05eecad8cd97 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:01.394198 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fb77f03d-75d0-477e-a47e-05eecad8cd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:01.501365 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 672/1327] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:01.791244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c1835-eb01-47dc-8c2a-4ae38c949b8d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:01.792920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-619c1835-eb01-47dc-8c2a-4ae38c949b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:01.856480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:01.901512 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c1835-eb01-47dc-8c2a-4ae38c949b8d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:01.976552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a2861-a250-4879-9727-dde517dc2abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:01.977164 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a2861-a250-4879-9727-dde517dc2abf service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.977441 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a2861-a250-4879-9727-dde517dc2abf service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.977705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a2861-a250-4879-9727-dde517dc2abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.979298 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-5a8a2861-a250-4879-9727-dde517dc2abf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:02.092179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:02.094435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a2861-a250-4879-9727-dde517dc2abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:02.095914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a2861-a250-4879-9727-dde517dc2abf service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:02.115200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 656/1328] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:04.102892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e02d4-0142-4020-b2d7-84d5b9c68b12 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:04.103449 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e02d4-0142-4020-b2d7-84d5b9c68b12 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:04.103566 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e02d4-0142-4020-b2d7-84d5b9c68b12 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:04.104884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 673/1329] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:04.110458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84499516-9271-43b1-80f7-82a495f261cb None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:04.110877 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84499516-9271-43b1-80f7-82a495f261cb None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:04.111013 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84499516-9271-43b1-80f7-82a495f261cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:04.182818 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-84499516-9271-43b1-80f7-82a495f261cb None None] MFA Rules not processed for user `e0ff5ce14df94fc4a1f5f037539e037c`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:04.184047 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-84499516-9271-43b1-80f7-82a495f261cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:04.322752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 657/1330] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:40:04 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:04.611800 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e02d4-0142-4020-b2d7-84d5b9c68b12 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:04.613403 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ef3e02d4-0142-4020-b2d7-84d5b9c68b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:04.645275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:04.722432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e02d4-0142-4020-b2d7-84d5b9c68b12 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:04.804223 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37913b0b-b23f-4cb8-ad60-bb28ccfa7323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:04.804924 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37913b0b-b23f-4cb8-ad60-bb28ccfa7323 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:04.805202 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37913b0b-b23f-4cb8-ad60-bb28ccfa7323 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:04.805502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37913b0b-b23f-4cb8-ad60-bb28ccfa7323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:04.807443 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-37913b0b-b23f-4cb8-ad60-bb28ccfa7323 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:04.895695 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:04.931704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37913b0b-b23f-4cb8-ad60-bb28ccfa7323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:04.933197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37913b0b-b23f-4cb8-ad60-bb28ccfa7323 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:04.953255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 674/1331] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:04 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:07.366123 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38fea1-a67b-4695-8aba-e063de655a97 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:07.366721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38fea1-a67b-4695-8aba-e063de655a97 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:07.366721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38fea1-a67b-4695-8aba-e063de655a97 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:07.367967 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 658/1332] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:40:07.373608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b37bb-ea94-4578-b7b8-454bc872c0f0 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:07.374082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b37bb-ea94-4578-b7b8-454bc872c0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:07.374184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b37bb-ea94-4578-b7b8-454bc872c0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:07.444698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c28b37bb-ea94-4578-b7b8-454bc872c0f0 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:07.445935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c28b37bb-ea94-4578-b7b8-454bc872c0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:07.549887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 675/1333] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:07.828554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38fea1-a67b-4695-8aba-e063de655a97 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:07.830083 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-9a38fea1-a67b-4695-8aba-e063de655a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:07.901644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:07.937423 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38fea1-a67b-4695-8aba-e063de655a97 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:08.012059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5760f-9a0b-4868-8d00-e480581b93b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:08.012690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5760f-9a0b-4868-8d00-e480581b93b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:08.012963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5760f-9a0b-4868-8d00-e480581b93b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:08.013230 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5760f-9a0b-4868-8d00-e480581b93b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:08.014809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-80f5760f-9a0b-4868-8d00-e480581b93b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:08.123653 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5760f-9a0b-4868-8d00-e480581b93b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:08.125132 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5760f-9a0b-4868-8d00-e480581b93b8 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:08.135714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:08.144160 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 659/1334] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:10.127134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbf48e-0872-4b80-bd33-e04bb9bd03fd None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.127673 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbf48e-0872-4b80-bd33-e04bb9bd03fd None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.127673 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbf48e-0872-4b80-bd33-e04bb9bd03fd None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.129085 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 676/1335] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:10.134670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec1286-aede-4dd4-8839-bbcbc9f7da44 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.135104 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec1286-aede-4dd4-8839-bbcbc9f7da44 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.135209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec1286-aede-4dd4-8839-bbcbc9f7da44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.202000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-35ec1286-aede-4dd4-8839-bbcbc9f7da44 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:10.203226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-35ec1286-aede-4dd4-8839-bbcbc9f7da44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:10.300367 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 660/1336] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:10.587159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbf48e-0872-4b80-bd33-e04bb9bd03fd None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:10.588779 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-47fbf48e-0872-4b80-bd33-e04bb9bd03fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:10.629252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:10.695920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbf48e-0872-4b80-bd33-e04bb9bd03fd None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:10.769041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72ddda-fa13-4ec5-a20e-87c348f9c603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:10.769644 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72ddda-fa13-4ec5-a20e-87c348f9c603 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.769922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72ddda-fa13-4ec5-a20e-87c348f9c603 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.770191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72ddda-fa13-4ec5-a20e-87c348f9c603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.771806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7a72ddda-fa13-4ec5-a20e-87c348f9c603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:10.853698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:10.882958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72ddda-fa13-4ec5-a20e-87c348f9c603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:10.884881 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72ddda-fa13-4ec5-a20e-87c348f9c603 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:10.905108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 677/1337] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:13.494315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2177b-3906-4ad3-b534-897db47535da None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:13.494881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2177b-3906-4ad3-b534-897db47535da None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:13.494881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2177b-3906-4ad3-b534-897db47535da None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:13.496363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 661/1338] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:13.502219 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49e173-4ab5-4b9d-8830-5c1e87488033 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:13.506923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49e173-4ab5-4b9d-8830-5c1e87488033 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:13.506923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49e173-4ab5-4b9d-8830-5c1e87488033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:13.573805 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-da49e173-4ab5-4b9d-8830-5c1e87488033 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:13.575098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-da49e173-4ab5-4b9d-8830-5c1e87488033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:13.677730 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 678/1339] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:13.962312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2177b-3906-4ad3-b534-897db47535da None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:13.963866 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-15b2177b-3906-4ad3-b534-897db47535da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:14.044515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:14.075810 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2177b-3906-4ad3-b534-897db47535da None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:14.150514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8146c8-cf11-440b-8f60-4d941eee1fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:14.151034 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8146c8-cf11-440b-8f60-4d941eee1fbd service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:14.151309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8146c8-cf11-440b-8f60-4d941eee1fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:14.151601 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8146c8-cf11-440b-8f60-4d941eee1fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:14.153216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6d8146c8-cf11-440b-8f60-4d941eee1fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:14.265498 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8146c8-cf11-440b-8f60-4d941eee1fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:14.267208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8146c8-cf11-440b-8f60-4d941eee1fbd service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:14.285783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 662/1340] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:18.835305 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971378d0-97fc-4175-b9e8-e7796865ebbe None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:18.835853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971378d0-97fc-4175-b9e8-e7796865ebbe None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:18.835853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971378d0-97fc-4175-b9e8-e7796865ebbe None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:18.838390 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 679/1341] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:18.842960 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f77d0-76e4-4e3f-8363-50cf447585e3 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:18.843424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f77d0-76e4-4e3f-8363-50cf447585e3 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:18.843518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f77d0-76e4-4e3f-8363-50cf447585e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:18.849003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:18.917220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-309f77d0-76e4-4e3f-8363-50cf447585e3 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:18.918510 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-309f77d0-76e4-4e3f-8363-50cf447585e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:19.023127 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 663/1342] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:19.310618 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971378d0-97fc-4175-b9e8-e7796865ebbe None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:19.312285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-971378d0-97fc-4175-b9e8-e7796865ebbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:19.365551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:19.424021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971378d0-97fc-4175-b9e8-e7796865ebbe None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:19.501055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d08c3b-3460-415b-a2e3-4e1d4f933322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:19.501665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d08c3b-3460-415b-a2e3-4e1d4f933322 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:19.501949 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d08c3b-3460-415b-a2e3-4e1d4f933322 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:19.502223 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d08c3b-3460-415b-a2e3-4e1d4f933322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:19.503865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b9d08c3b-3460-415b-a2e3-4e1d4f933322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:19.608018 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:19.624661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d08c3b-3460-415b-a2e3-4e1d4f933322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:19.634660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d08c3b-3460-415b-a2e3-4e1d4f933322 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:19.645242 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 680/1343] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:22.694416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624736cc-4aa8-4818-844d-7abd524066c8 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:22.694980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624736cc-4aa8-4818-844d-7abd524066c8 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:22.694980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624736cc-4aa8-4818-844d-7abd524066c8 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:22.696353 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 664/1344] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:40:22.702017 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebbd8f-2ee9-4265-b966-ac58aaf9c1da None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:22.702482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebbd8f-2ee9-4265-b966-ac58aaf9c1da None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:22.702578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebbd8f-2ee9-4265-b966-ac58aaf9c1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:22.773747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a2ebbd8f-2ee9-4265-b966-ac58aaf9c1da None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:22.774964 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a2ebbd8f-2ee9-4265-b966-ac58aaf9c1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:22.879771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 681/1345] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:40:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:23.169216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624736cc-4aa8-4818-844d-7abd524066c8 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:23.170759 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-624736cc-4aa8-4818-844d-7abd524066c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:23.262008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:23.280642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624736cc-4aa8-4818-844d-7abd524066c8 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:23.361324 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d8e14-e67c-4d3b-86ca-24a645381496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:23.361936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d8e14-e67c-4d3b-86ca-24a645381496 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.362216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d8e14-e67c-4d3b-86ca-24a645381496 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.362486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d8e14-e67c-4d3b-86ca-24a645381496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.364190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-446d8e14-e67c-4d3b-86ca-24a645381496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:23.494444 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d8e14-e67c-4d3b-86ca-24a645381496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:23.495908 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d8e14-e67c-4d3b-86ca-24a645381496 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:23.516476 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 665/1346] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:27.674399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c1097-4791-4664-9472-a557ace179c8 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:27.674954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c1097-4791-4664-9472-a557ace179c8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:27.674954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c1097-4791-4664-9472-a557ace179c8 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:27.676339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 682/1347] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:27.682611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c83e9c-6f91-4c48-bc03-b03fd4627952 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:27.683126 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c83e9c-6f91-4c48-bc03-b03fd4627952 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:27.683209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c83e9c-6f91-4c48-bc03-b03fd4627952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:27.700017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:27.760945 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-24c83e9c-6f91-4c48-bc03-b03fd4627952 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:27.762217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-24c83e9c-6f91-4c48-bc03-b03fd4627952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:27.868826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 666/1348] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:28.154209 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c1097-4791-4664-9472-a557ace179c8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:28.155757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-870c1097-4791-4664-9472-a557ace179c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:28.216813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:28.267454 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c1097-4791-4664-9472-a557ace179c8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:28.342886 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afdaa3-f1e8-4292-87f6-7e22f7365583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:28.343490 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afdaa3-f1e8-4292-87f6-7e22f7365583 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:28.343764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afdaa3-f1e8-4292-87f6-7e22f7365583 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:28.344035 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afdaa3-f1e8-4292-87f6-7e22f7365583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:28.345671 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a2afdaa3-f1e8-4292-87f6-7e22f7365583 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:28.450263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:28.453771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afdaa3-f1e8-4292-87f6-7e22f7365583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:28.455220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afdaa3-f1e8-4292-87f6-7e22f7365583 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:28.473471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 683/1349] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:33.483029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1011b9-6787-4f1c-8b76-54e42c725cf5 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:33.483029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1011b9-6787-4f1c-8b76-54e42c725cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:33.483029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1011b9-6787-4f1c-8b76-54e42c725cf5 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:33.483029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 667/1350] 10.222.0.70 () {54 vars in 921 bytes} [Wed Aug 24 16:40:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:33.490009 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4e4bf-f8bc-48c2-8c1c-4b58a3a4190a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:33.490009 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4e4bf-f8bc-48c2-8c1c-4b58a3a4190a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:33.490517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4e4bf-f8bc-48c2-8c1c-4b58a3a4190a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:33.595560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-28b4e4bf-f8bc-48c2-8c1c-4b58a3a4190a None None] MFA Rules not processed for user `fc4c43c85a5d49979ba803fdb1041c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:33.596822 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-28b4e4bf-f8bc-48c2-8c1c-4b58a3a4190a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:33.779450 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 684/1351] 10.222.0.70 () {58 vars in 1018 bytes} [Wed Aug 24 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:37.866575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=59 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:37.925385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9fcd7-91a5-47be-811d-6f9f2f03eea2 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.925643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9fcd7-91a5-47be-811d-6f9f2f03eea2 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.925988 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9fcd7-91a5-47be-811d-6f9f2f03eea2 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.927421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 668/1352] 10.222.0.70 () {54 vars in 921 bytes} [Wed Aug 24 16:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:37.933054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09029eb6-feb1-4fad-b631-a5dc5755d623 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.933557 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09029eb6-feb1-4fad-b631-a5dc5755d623 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.933647 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09029eb6-feb1-4fad-b631-a5dc5755d623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.992507 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:38.016752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-09029eb6-feb1-4fad-b631-a5dc5755d623 None None] MFA Rules not processed for user `fc4c43c85a5d49979ba803fdb1041c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:38.018035 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-09029eb6-feb1-4fad-b631-a5dc5755d623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:38.134468 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 685/1353] 10.222.0.70 () {58 vars in 1018 bytes} [Wed Aug 24 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:39.627234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8f9a6-0f51-40e1-b666-303ba5694514 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:39.627791 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8f9a6-0f51-40e1-b666-303ba5694514 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:39.627968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8f9a6-0f51-40e1-b666-303ba5694514 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:39.629436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 669/1354] 10.222.0.70 () {54 vars in 921 bytes} [Wed Aug 24 16:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:39.635856 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6850bb-c264-49a1-9f7a-0b3d9e69cd7a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:39.636311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6850bb-c264-49a1-9f7a-0b3d9e69cd7a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:39.636603 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6850bb-c264-49a1-9f7a-0b3d9e69cd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:39.721674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7f6850bb-c264-49a1-9f7a-0b3d9e69cd7a None None] MFA Rules not processed for user `fc4c43c85a5d49979ba803fdb1041c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:39.723019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7f6850bb-c264-49a1-9f7a-0b3d9e69cd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:39.765860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:39.841127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 686/1355] 10.222.0.70 () {58 vars in 1018 bytes} [Wed Aug 24 16:40:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:43.041422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029eb32-1785-4dfa-82bf-8865daf89d2a None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:43.042006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029eb32-1785-4dfa-82bf-8865daf89d2a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:43.042006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029eb32-1785-4dfa-82bf-8865daf89d2a None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:43.043931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 670/1356] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:43.048816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbb730-1d89-4675-b042-e2ffcf08a4b2 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:43.049271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbb730-1d89-4675-b042-e2ffcf08a4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:43.049360 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbb730-1d89-4675-b042-e2ffcf08a4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:43.136761 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-55cbb730-1d89-4675-b042-e2ffcf08a4b2 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:43.138248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-55cbb730-1d89-4675-b042-e2ffcf08a4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:43.271346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 687/1357] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:43.585976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbfb1d-23aa-4345-8dbe-7849d8ee3179 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:43.586670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbfb1d-23aa-4345-8dbe-7849d8ee3179 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:43.587363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbfb1d-23aa-4345-8dbe-7849d8ee3179 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:43.590193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 671/1358] 10.222.0.70 () {54 vars in 962 bytes} [Wed Aug 24 16:40:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:43.598331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ff2fd-8a31-453d-a112-8684f71c8bf4 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:43.598745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ff2fd-8a31-453d-a112-8684f71c8bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:43.599130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ff2fd-8a31-453d-a112-8684f71c8bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:43.617287 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:43.683627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c59ff2fd-8a31-453d-a112-8684f71c8bf4 None None] MFA Rules not processed for user `e333fca7790244f6a4ebceabb55faba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:43.685036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c59ff2fd-8a31-453d-a112-8684f71c8bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:43.888553 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 688/1359] 10.222.0.70 () {58 vars in 1059 bytes} [Wed Aug 24 16:40:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:43.889656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbfb1d-23aa-4345-8dbe-7849d8ee3179 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:43.891160 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a4cbfb1d-23aa-4345-8dbe-7849d8ee3179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:44.070918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbfb1d-23aa-4345-8dbe-7849d8ee3179 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:44.150723 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:44.210691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e6e29-ffcf-4316-8e9a-ddcca768a26c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:44.211300 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e6e29-ffcf-4316-8e9a-ddcca768a26c service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:44.211580 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e6e29-ffcf-4316-8e9a-ddcca768a26c service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:44.211865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e6e29-ffcf-4316-8e9a-ddcca768a26c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:44.213869 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c61e6e29-ffcf-4316-8e9a-ddcca768a26c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:44.325668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e6e29-ffcf-4316-8e9a-ddcca768a26c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:44.327208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e6e29-ffcf-4316-8e9a-ddcca768a26c service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:44.347768 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 672/1360] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:46.533059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b0047-5239-41b1-9e1c-d5f375ea5cf9 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:46.533693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b0047-5239-41b1-9e1c-d5f375ea5cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:46.533693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b0047-5239-41b1-9e1c-d5f375ea5cf9 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:46.534954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 689/1361] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:46.543046 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d5c12-9acf-4c77-bdef-7d7d7a37ecbd None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:46.543452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d5c12-9acf-4c77-bdef-7d7d7a37ecbd None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:46.543452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d5c12-9acf-4c77-bdef-7d7d7a37ecbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:46.616852 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-2d3d5c12-9acf-4c77-bdef-7d7d7a37ecbd None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:46.618062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2d3d5c12-9acf-4c77-bdef-7d7d7a37ecbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:46.659915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:46.726822 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 673/1362] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:47.025392 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b0047-5239-41b1-9e1c-d5f375ea5cf9 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:47.026921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c37b0047-5239-41b1-9e1c-d5f375ea5cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:47.103381 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:47.198130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b0047-5239-41b1-9e1c-d5f375ea5cf9 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:47.323023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc1627-cab8-46f1-b1e7-ccb8cef74304 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:47.323637 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1627-cab8-46f1-b1e7-ccb8cef74304 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:47.323923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1627-cab8-46f1-b1e7-ccb8cef74304 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:47.324190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1627-cab8-46f1-b1e7-ccb8cef74304 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:47.325970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d5bc1627-cab8-46f1-b1e7-ccb8cef74304 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:47.351429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:47.431607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc1627-cab8-46f1-b1e7-ccb8cef74304 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:47.433472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc1627-cab8-46f1-b1e7-ccb8cef74304 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:47.453983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 690/1363] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:49.856993 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e42a2-058b-4829-849e-c4baed63bea8 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:49.857513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e42a2-058b-4829-849e-c4baed63bea8 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:49.857619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e42a2-058b-4829-849e-c4baed63bea8 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:49.859058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 674/1364] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:49.865863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b5286-1caa-4a72-abf1-c49144f4fe3d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:49.866329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b5286-1caa-4a72-abf1-c49144f4fe3d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:49.866412 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b5286-1caa-4a72-abf1-c49144f4fe3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:49.937154 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-375b5286-1caa-4a72-abf1-c49144f4fe3d None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:49.938384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-375b5286-1caa-4a72-abf1-c49144f4fe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:50.039581 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:50.044447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 691/1365] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:50.335601 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcff5e-778e-4a79-b805-0a959d7eacf7 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:50.335882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcff5e-778e-4a79-b805-0a959d7eacf7 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:50.336337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcff5e-778e-4a79-b805-0a959d7eacf7 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:50.338604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 675/1366] 10.222.0.70 () {54 vars in 962 bytes} [Wed Aug 24 16:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:50.345394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1675d7a-a0d8-4852-af24-c96016a96b2a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:50.345675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1675d7a-a0d8-4852-af24-c96016a96b2a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:50.345934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1675d7a-a0d8-4852-af24-c96016a96b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:50.428941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-d1675d7a-a0d8-4852-af24-c96016a96b2a None None] MFA Rules not processed for user `e333fca7790244f6a4ebceabb55faba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:50.430313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d1675d7a-a0d8-4852-af24-c96016a96b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:50.567020 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 692/1367] 10.222.0.70 () {58 vars in 1059 bytes} [Wed Aug 24 16:40:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:50.576381 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcff5e-778e-4a79-b805-0a959d7eacf7 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:50.578202 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-05bcff5e-778e-4a79-b805-0a959d7eacf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:50.736583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcff5e-778e-4a79-b805-0a959d7eacf7 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:50.759735 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:50.870861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971e183-0d94-4782-99af-2581f42a4a7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:50.871495 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971e183-0d94-4782-99af-2581f42a4a7a service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:50.871753 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971e183-0d94-4782-99af-2581f42a4a7a service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:50.872027 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971e183-0d94-4782-99af-2581f42a4a7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:50.873692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1971e183-0d94-4782-99af-2581f42a4a7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:50.969249 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971e183-0d94-4782-99af-2581f42a4a7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:50.972461 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971e183-0d94-4782-99af-2581f42a4a7a service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:50.991454 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 676/1368] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:51.441335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00ea04-ae80-4e1a-8395-c19018cd1319 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:51.441883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00ea04-ae80-4e1a-8395-c19018cd1319 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:51.441883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00ea04-ae80-4e1a-8395-c19018cd1319 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:51.443412 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 693/1369] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:51.449052 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149a4f8-a85d-4e25-9ca6-e0285bb04304 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:51.449354 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149a4f8-a85d-4e25-9ca6-e0285bb04304 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:51.449568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149a4f8-a85d-4e25-9ca6-e0285bb04304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:51.484735 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:51.523364 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-8149a4f8-a85d-4e25-9ca6-e0285bb04304 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:51.525318 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-8149a4f8-a85d-4e25-9ca6-e0285bb04304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:51.628776 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 677/1370] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:52.217253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87009597-54e8-4dcd-a482-d71ea2601fd6 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:52.217253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87009597-54e8-4dcd-a482-d71ea2601fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:52.217253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87009597-54e8-4dcd-a482-d71ea2601fd6 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:52.220019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 694/1371] 10.222.0.70 () {54 vars in 964 bytes} [Wed Aug 24 16:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:52.222892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840c294-558f-45b0-b4ab-e427c07e91cf None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:52.223379 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840c294-558f-45b0-b4ab-e427c07e91cf None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:52.223537 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840c294-558f-45b0-b4ab-e427c07e91cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:52.306719 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-3840c294-558f-45b0-b4ab-e427c07e91cf None None] MFA Rules not processed for user `f08140bcbd1e43dda77d4387666f9085`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:52.308182 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3840c294-558f-45b0-b4ab-e427c07e91cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:52.360026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:52.391494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 678/1372] 10.222.0.70 () {58 vars in 1061 bytes} [Wed Aug 24 16:40:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:52.399603 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87009597-54e8-4dcd-a482-d71ea2601fd6 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:52.401317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-87009597-54e8-4dcd-a482-d71ea2601fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:52.454185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:52.511513 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87009597-54e8-4dcd-a482-d71ea2601fd6 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:52.593391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27eafd-c707-4864-8209-f818b5c10d1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:52.594004 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27eafd-c707-4864-8209-f818b5c10d1f service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:52.594286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27eafd-c707-4864-8209-f818b5c10d1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:52.594722 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27eafd-c707-4864-8209-f818b5c10d1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:52.596362 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6c27eafd-c707-4864-8209-f818b5c10d1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:52.701371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:52.718541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27eafd-c707-4864-8209-f818b5c10d1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:52.719999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27eafd-c707-4864-8209-f818b5c10d1f service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:52.739696 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 695/1373] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:40:53.139265 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38004c57-e076-46fb-a49e-aacb68fef21d None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:53.139545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38004c57-e076-46fb-a49e-aacb68fef21d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:53.139817 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38004c57-e076-46fb-a49e-aacb68fef21d None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:53.141196 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 679/1374] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:53.146492 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ffc54-474a-4b59-aaff-40cc311e3e41 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:53.146787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ffc54-474a-4b59-aaff-40cc311e3e41 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:53.147098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ffc54-474a-4b59-aaff-40cc311e3e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:53.214632 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fe0ffc54-474a-4b59-aaff-40cc311e3e41 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:53.215759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fe0ffc54-474a-4b59-aaff-40cc311e3e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:53.315533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 696/1375] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:53.608686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38004c57-e076-46fb-a49e-aacb68fef21d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:53.610236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-38004c57-e076-46fb-a49e-aacb68fef21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:53.762646 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38004c57-e076-46fb-a49e-aacb68fef21d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:53.818368 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:53.881718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e76a5d-a5b7-4135-b7fc-557a935c6867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:53.882333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e76a5d-a5b7-4135-b7fc-557a935c6867 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:53.882615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e76a5d-a5b7-4135-b7fc-557a935c6867 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:53.882890 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e76a5d-a5b7-4135-b7fc-557a935c6867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:53.885049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-81e76a5d-a5b7-4135-b7fc-557a935c6867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:53.975662 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e76a5d-a5b7-4135-b7fc-557a935c6867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:53.977312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e76a5d-a5b7-4135-b7fc-557a935c6867 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:53.995143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 680/1376] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:56.086165 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820e40f-92d1-4ebd-b3a9-98b675e4a0b1 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:56.086681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820e40f-92d1-4ebd-b3a9-98b675e4a0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:56.086681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820e40f-92d1-4ebd-b3a9-98b675e4a0b1 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:56.087995 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 697/1377] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:40:56.093506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3e5d3-fcc9-43b7-80af-cbfceefa7f2c None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:56.093950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3e5d3-fcc9-43b7-80af-cbfceefa7f2c None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:56.094048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3e5d3-fcc9-43b7-80af-cbfceefa7f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:56.162539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-7ab3e5d3-fcc9-43b7-80af-cbfceefa7f2c None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:56.163756 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7ab3e5d3-fcc9-43b7-80af-cbfceefa7f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:56.171322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:56.271722 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 681/1378] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:56.568871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820e40f-92d1-4ebd-b3a9-98b675e4a0b1 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:56.570636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8820e40f-92d1-4ebd-b3a9-98b675e4a0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:56.637355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:56.732319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820e40f-92d1-4ebd-b3a9-98b675e4a0b1 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:56.862541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac9918-a5f7-4511-b5bb-ea25d8120f0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:56.863142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac9918-a5f7-4511-b5bb-ea25d8120f0f service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:56.863420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac9918-a5f7-4511-b5bb-ea25d8120f0f service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:56.863686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac9918-a5f7-4511-b5bb-ea25d8120f0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:56.865380 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-14ac9918-a5f7-4511-b5bb-ea25d8120f0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:56.885685 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:56.965031 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac9918-a5f7-4511-b5bb-ea25d8120f0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:56.966842 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac9918-a5f7-4511-b5bb-ea25d8120f0f service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:56.986243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 698/1379] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:59.078028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fd3df-0231-4be9-9b54-1e5bc47e15e1 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:59.078561 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fd3df-0231-4be9-9b54-1e5bc47e15e1 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:59.078561 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fd3df-0231-4be9-9b54-1e5bc47e15e1 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:59.080061 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 682/1380] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:40:59.085389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d1684-bf2f-46ef-8fea-34e0f53759e9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:59.085848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d1684-bf2f-46ef-8fea-34e0f53759e9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:59.085944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d1684-bf2f-46ef-8fea-34e0f53759e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:59.158340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-d46d1684-bf2f-46ef-8fea-34e0f53759e9 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:59.159547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d46d1684-bf2f-46ef-8fea-34e0f53759e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:59.264536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:59.272382 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 699/1381] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:59.562600 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fd3df-0231-4be9-9b54-1e5bc47e15e1 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:59.566542 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-8b5fd3df-0231-4be9-9b54-1e5bc47e15e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:59.708871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:59.719668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fd3df-0231-4be9-9b54-1e5bc47e15e1 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:59.859748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89527cc3-205f-4015-a6e6-dcec8a9afd66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:59.860979 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89527cc3-205f-4015-a6e6-dcec8a9afd66 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:59.861475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89527cc3-205f-4015-a6e6-dcec8a9afd66 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:59.861967 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89527cc3-205f-4015-a6e6-dcec8a9afd66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:59.863957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-89527cc3-205f-4015-a6e6-dcec8a9afd66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:59.973828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89527cc3-205f-4015-a6e6-dcec8a9afd66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:59.975307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89527cc3-205f-4015-a6e6-dcec8a9afd66 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:59.986388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:59.996603 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 683/1382] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:02.421557 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8b896-7b94-46cf-b90b-24e8fc851ffb None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:02.422241 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8b896-7b94-46cf-b90b-24e8fc851ffb None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:02.422241 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8b896-7b94-46cf-b90b-24e8fc851ffb None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:02.423494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 700/1383] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:02.430558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce2eea-be00-40d7-bdd4-f92d6ffc038d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:02.431020 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce2eea-be00-40d7-bdd4-f92d6ffc038d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:02.431118 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce2eea-be00-40d7-bdd4-f92d6ffc038d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:02.506886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-03ce2eea-be00-40d7-bdd4-f92d6ffc038d None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:02.508103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-03ce2eea-be00-40d7-bdd4-f92d6ffc038d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:02.614559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 684/1384] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:02.914450 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8b896-7b94-46cf-b90b-24e8fc851ffb None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:02.916069 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7fa8b896-7b94-46cf-b90b-24e8fc851ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:03.091545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8b896-7b94-46cf-b90b-24e8fc851ffb None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:03.163151 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:03.211460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724c8e7-25b9-47a1-a688-8b7abe6b2c98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:03.212070 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724c8e7-25b9-47a1-a688-8b7abe6b2c98 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:03.212350 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724c8e7-25b9-47a1-a688-8b7abe6b2c98 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:03.212654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724c8e7-25b9-47a1-a688-8b7abe6b2c98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:03.214300 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8724c8e7-25b9-47a1-a688-8b7abe6b2c98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:03.307888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724c8e7-25b9-47a1-a688-8b7abe6b2c98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:03.309531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724c8e7-25b9-47a1-a688-8b7abe6b2c98 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:03.328820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 701/1385] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:41:05.272192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4119fa-ea8e-46aa-93a0-7996a90596ce None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:05.272808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4119fa-ea8e-46aa-93a0-7996a90596ce None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:05.272808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4119fa-ea8e-46aa-93a0-7996a90596ce None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:05.274793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 685/1386] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:41:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:41:05.280961 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d846a-d4ca-43a7-b3ce-fc273a21133e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:05.281506 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d846a-d4ca-43a7-b3ce-fc273a21133e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:05.281506 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d846a-d4ca-43a7-b3ce-fc273a21133e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:05.350140 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-858d846a-d4ca-43a7-b3ce-fc273a21133e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:05.351405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-858d846a-d4ca-43a7-b3ce-fc273a21133e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:05.386963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:05.463632 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 702/1387] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:05.753028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4119fa-ea8e-46aa-93a0-7996a90596ce None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:05.754660 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-dc4119fa-ea8e-46aa-93a0-7996a90596ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:05.848147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:05.931684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4119fa-ea8e-46aa-93a0-7996a90596ce None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:06.069242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb81d9c-e914-44ef-9229-aa14433cf910 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:06.069877 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb81d9c-e914-44ef-9229-aa14433cf910 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:06.070198 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb81d9c-e914-44ef-9229-aa14433cf910 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:06.070482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb81d9c-e914-44ef-9229-aa14433cf910 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:06.072159 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-aeb81d9c-e914-44ef-9229-aa14433cf910 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:06.114423 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:06.171826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb81d9c-e914-44ef-9229-aa14433cf910 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:06.173499 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb81d9c-e914-44ef-9229-aa14433cf910 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:06.196093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 686/1388] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:08.144326 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4b448-8d91-4093-820a-027ea7fc54b7 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:08.144932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4b448-8d91-4093-820a-027ea7fc54b7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:08.144932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4b448-8d91-4093-820a-027ea7fc54b7 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:08.146253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 703/1389] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:08.152289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d7fec-1d06-4dc4-8c81-46998211c8fa None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:08.152781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d7fec-1d06-4dc4-8c81-46998211c8fa None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:08.152888 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d7fec-1d06-4dc4-8c81-46998211c8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:08.220611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-ac7d7fec-1d06-4dc4-8c81-46998211c8fa None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:08.221765 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ac7d7fec-1d06-4dc4-8c81-46998211c8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:08.321024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 687/1390] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:08.444124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493060a8-ba04-4202-b086-3bd296613f4b None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:08.444453 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493060a8-ba04-4202-b086-3bd296613f4b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:08.444764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493060a8-ba04-4202-b086-3bd296613f4b None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:08.446151 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 704/1391] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:08.451620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9c599-ef0f-48cf-a74a-7706ddd77654 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:08.451912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9c599-ef0f-48cf-a74a-7706ddd77654 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:08.452192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9c599-ef0f-48cf-a74a-7706ddd77654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:08.468552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:08.520387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-e4a9c599-ef0f-48cf-a74a-7706ddd77654 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:08.521623 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e4a9c599-ef0f-48cf-a74a-7706ddd77654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:08.621549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 688/1392] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:08.626181 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e9083-c28f-4b14-b8b3-4ee0a8eaede3 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:08.626464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e9083-c28f-4b14-b8b3-4ee0a8eaede3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:08.626743 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e9083-c28f-4b14-b8b3-4ee0a8eaede3 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:08.628056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 705/1393] 10.222.0.70 () {54 vars in 967 bytes} [Wed Aug 24 16:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:41:08.633376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5979f-c83d-4516-ba18-0c2fa6421c45 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:08.633669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5979f-c83d-4516-ba18-0c2fa6421c45 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:08.633945 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5979f-c83d-4516-ba18-0c2fa6421c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:08.712503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-fcd5979f-c83d-4516-ba18-0c2fa6421c45 None None] MFA Rules not processed for user `56418ff737504181a59714aad9f1c375`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:08.713621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-fcd5979f-c83d-4516-ba18-0c2fa6421c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:08.750598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:08.793820 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 689/1394] 10.222.0.70 () {58 vars in 1064 bytes} [Wed Aug 24 16:41:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:41:08.801689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e9083-c28f-4b14-b8b3-4ee0a8eaede3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:08.803203 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-118e9083-c28f-4b14-b8b3-4ee0a8eaede3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:08.908524 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e9083-c28f-4b14-b8b3-4ee0a8eaede3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:08.914396 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5979f-c83d-4516-ba18-0c2fa6421c45 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:08.915602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-fcd5979f-c83d-4516-ba18-0c2fa6421c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:08.970606 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:08.986628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573a5bb-503b-4954-9d65-4ccb28079a27 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56418ff737504181a59714aad9f1c375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:08.987236 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573a5bb-503b-4954-9d65-4ccb28079a27 service glance] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:08.987515 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573a5bb-503b-4954-9d65-4ccb28079a27 service glance] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:08.987789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573a5bb-503b-4954-9d65-4ccb28079a27 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:08.989420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7573a5bb-503b-4954-9d65-4ccb28079a27 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:09.036465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5979f-c83d-4516-ba18-0c2fa6421c45 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:09.101037 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573a5bb-503b-4954-9d65-4ccb28079a27 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:09.101995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:09.103025 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573a5bb-503b-4954-9d65-4ccb28079a27 service glance] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:09.121683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 706/1395] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:09.157915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032c909-bad7-446c-bec2-d554197101f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:09.158558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032c909-bad7-446c-bec2-d554197101f2 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:09.158861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032c909-bad7-446c-bec2-d554197101f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:09.159154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032c909-bad7-446c-bec2-d554197101f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:09.160811 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a032c909-bad7-446c-bec2-d554197101f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:09.255774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032c909-bad7-446c-bec2-d554197101f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:09.257337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032c909-bad7-446c-bec2-d554197101f2 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:09.276136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 690/1396] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:41:10.016360 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd70ce-a2fe-4a42-95f9-5a2d15d6309a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:10.016872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd70ce-a2fe-4a42-95f9-5a2d15d6309a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:10.017100 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd70ce-a2fe-4a42-95f9-5a2d15d6309a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:10.085000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-95dd70ce-a2fe-4a42-95f9-5a2d15d6309a None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:10.086204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-95dd70ce-a2fe-4a42-95f9-5a2d15d6309a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:10.126373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:10.171833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 707/1397] 10.222.0.70 () {58 vars in 1009 bytes} [Wed Aug 24 16:41:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:10.219259 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfa312-be64-4661-b70b-1f59d38fd949 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:10.219741 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfa312-be64-4661-b70b-1f59d38fd949 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:10.219835 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfa312-be64-4661-b70b-1f59d38fd949 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:10.221409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 691/1398] 10.222.0.70 () {54 vars in 943 bytes} [Wed Aug 24 16:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:10.228721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd1271-d49b-44b6-8c04-4801f3b49f6b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:10.229333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd1271-d49b-44b6-8c04-4801f3b49f6b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:10.229747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd1271-d49b-44b6-8c04-4801f3b49f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:10.309675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-affd1271-d49b-44b6-8c04-4801f3b49f6b None None] MFA Rules not processed for user `b8ec0b1e5f4d434bb725297f3f867ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:10.310787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-affd1271-d49b-44b6-8c04-4801f3b49f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:10.423023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 708/1399] 10.222.0.70 () {58 vars in 1040 bytes} [Wed Aug 24 16:41:10 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:41:10.432841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfa312-be64-4661-b70b-1f59d38fd949 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:10.434391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3abfa312-be64-4661-b70b-1f59d38fd949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:10.514397 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:10.579907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfa312-be64-4661-b70b-1f59d38fd949 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:10.691260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c954c23-9098-4685-941f-203cad00d3c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:10.691889 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c954c23-9098-4685-941f-203cad00d3c8 service swift] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:10.692151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c954c23-9098-4685-941f-203cad00d3c8 service swift] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:10.692457 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c954c23-9098-4685-941f-203cad00d3c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:10.694085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1c954c23-9098-4685-941f-203cad00d3c8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:10.756816 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:10.780356 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c954c23-9098-4685-941f-203cad00d3c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:10.781965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c954c23-9098-4685-941f-203cad00d3c8 service swift] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:10.791009 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 692/1400] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 16:41:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:41:11.378189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb8a9c-aa3b-477d-9d82-7dad649b8483 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:11.378741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb8a9c-aa3b-477d-9d82-7dad649b8483 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:11.378741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb8a9c-aa3b-477d-9d82-7dad649b8483 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:11.381325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 709/1401] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:11.386093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c07d63-249d-459a-a48a-584ca35e5f55 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:11.386586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c07d63-249d-459a-a48a-584ca35e5f55 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:11.386756 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c07d63-249d-459a-a48a-584ca35e5f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:11.459357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-03c07d63-249d-459a-a48a-584ca35e5f55 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:11.461151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-03c07d63-249d-459a-a48a-584ca35e5f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:11.582418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 693/1402] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:11.891221 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb8a9c-aa3b-477d-9d82-7dad649b8483 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:11.891221 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-71cb8a9c-aa3b-477d-9d82-7dad649b8483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:11.907277 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:12.061720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb8a9c-aa3b-477d-9d82-7dad649b8483 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:12.162457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:12.190851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d66aa-9b34-4e30-8fb9-b22ac1ce21de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:12.191874 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d66aa-9b34-4e30-8fb9-b22ac1ce21de service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:12.192222 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d66aa-9b34-4e30-8fb9-b22ac1ce21de service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:12.192623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d66aa-9b34-4e30-8fb9-b22ac1ce21de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:12.194436 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9d8d66aa-9b34-4e30-8fb9-b22ac1ce21de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:12.301834 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d66aa-9b34-4e30-8fb9-b22ac1ce21de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:12.303526 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d66aa-9b34-4e30-8fb9-b22ac1ce21de service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:12.332870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 710/1403] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:14.443089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccf723-a518-42fb-be7c-c4c00bff9287 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:14.443664 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccf723-a518-42fb-be7c-c4c00bff9287 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:14.443664 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccf723-a518-42fb-be7c-c4c00bff9287 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:14.445806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 694/1404] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:41:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:14.451369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ffb9e-a54d-4caa-bf5d-5a65f3821771 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:14.451814 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ffb9e-a54d-4caa-bf5d-5a65f3821771 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:14.451918 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ffb9e-a54d-4caa-bf5d-5a65f3821771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:14.530884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9c5ffb9e-a54d-4caa-bf5d-5a65f3821771 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:14.532167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9c5ffb9e-a54d-4caa-bf5d-5a65f3821771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:14.593032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:14.654831 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 711/1405] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:14.940336 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccf723-a518-42fb-be7c-c4c00bff9287 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:14.941984 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a7ccf723-a518-42fb-be7c-c4c00bff9287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:15.014699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:15.129618 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccf723-a518-42fb-be7c-c4c00bff9287 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:15.261619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e456d14-d3a8-4561-a4b9-67bbeca979fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:15.262418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e456d14-d3a8-4561-a4b9-67bbeca979fc service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:15.262734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e456d14-d3a8-4561-a4b9-67bbeca979fc service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:15.263014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e456d14-d3a8-4561-a4b9-67bbeca979fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:15.264818 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3e456d14-d3a8-4561-a4b9-67bbeca979fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:15.282744 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:15.386543 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e456d14-d3a8-4561-a4b9-67bbeca979fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:15.388093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e456d14-d3a8-4561-a4b9-67bbeca979fc service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:15.416609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 695/1406] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:17.967646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc0d10-d1bb-4ef9-b286-a73c7941ad56 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.968218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc0d10-d1bb-4ef9-b286-a73c7941ad56 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.968218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc0d10-d1bb-4ef9-b286-a73c7941ad56 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:17.969578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 712/1407] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:17.976968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2b607-1d78-408c-b3ea-fc783f589520 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.979323 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2b607-1d78-408c-b3ea-fc783f589520 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.979323 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2b607-1d78-408c-b3ea-fc783f589520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:18.046877 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-82c2b607-1d78-408c-b3ea-fc783f589520 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:18.048232 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-82c2b607-1d78-408c-b3ea-fc783f589520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:18.144228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:18.161250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 696/1408] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:18.464488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc0d10-d1bb-4ef9-b286-a73c7941ad56 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:18.466055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e7cc0d10-d1bb-4ef9-b286-a73c7941ad56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:18.641806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc0d10-d1bb-4ef9-b286-a73c7941ad56 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:18.680382 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:18.780929 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f70cbf-2cec-4944-8b2f-0b817852d570 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:18.781635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f70cbf-2cec-4944-8b2f-0b817852d570 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:18.781917 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f70cbf-2cec-4944-8b2f-0b817852d570 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:18.782186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f70cbf-2cec-4944-8b2f-0b817852d570 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:18.783836 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b9f70cbf-2cec-4944-8b2f-0b817852d570 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:18.885369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f70cbf-2cec-4944-8b2f-0b817852d570 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:18.887362 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f70cbf-2cec-4944-8b2f-0b817852d570 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:18.908855 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 713/1409] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:28.201319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:28.905654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=83 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:34.139416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4ac38-7b01-495f-aae3-1db08751fb73 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:34.139980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4ac38-7b01-495f-aae3-1db08751fb73 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:34.139980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4ac38-7b01-495f-aae3-1db08751fb73 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:34.141350 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 697/1410] 10.222.0.70 () {54 vars in 918 bytes} [Wed Aug 24 16:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:34.146541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483478a-8e27-4c27-b5e9-bfe805dbc58b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:34.146950 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483478a-8e27-4c27-b5e9-bfe805dbc58b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:34.147086 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483478a-8e27-4c27-b5e9-bfe805dbc58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:34.228512 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8483478a-8e27-4c27-b5e9-bfe805dbc58b None None] MFA Rules not processed for user `fc4c43c85a5d49979ba803fdb1041c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:34.229758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8483478a-8e27-4c27-b5e9-bfe805dbc58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:34.335630 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 714/1411] 10.222.0.70 () {58 vars in 1015 bytes} [Wed Aug 24 16:41:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:34.349913 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7a3f6-da9b-4cbb-83cd-d9e768d13030 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:34.350216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7a3f6-da9b-4cbb-83cd-d9e768d13030 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:34.350527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7a3f6-da9b-4cbb-83cd-d9e768d13030 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:34.351846 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 698/1412] 10.222.0.70 () {54 vars in 918 bytes} [Wed Aug 24 16:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:34.356468 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16430280-ab3e-4cf4-88a9-f0935cabb46f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:34.356758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16430280-ab3e-4cf4-88a9-f0935cabb46f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:34.357033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16430280-ab3e-4cf4-88a9-f0935cabb46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:34.429644 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:34.432537 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-16430280-ab3e-4cf4-88a9-f0935cabb46f None None] MFA Rules not processed for user `e333fca7790244f6a4ebceabb55faba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:34.433740 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-16430280-ab3e-4cf4-88a9-f0935cabb46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:34.549859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 715/1413] 10.222.0.70 () {58 vars in 1015 bytes} [Wed Aug 24 16:41:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:36.963489 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671925e-96cb-4a79-875e-58502aa61109 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:36.964029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671925e-96cb-4a79-875e-58502aa61109 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:36.964029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671925e-96cb-4a79-875e-58502aa61109 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:36.965386 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:36.970824 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7c348-fb6e-4790-b5b2-0242f8d50e58 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:36.971271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7c348-fb6e-4790-b5b2-0242f8d50e58 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:36.971370 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7c348-fb6e-4790-b5b2-0242f8d50e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:37.037953 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-31b7c348-fb6e-4790-b5b2-0242f8d50e58 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:37.039153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-31b7c348-fb6e-4790-b5b2-0242f8d50e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:37.093628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:37.136727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:37.427187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671925e-96cb-4a79-875e-58502aa61109 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:37.428782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b671925e-96cb-4a79-875e-58502aa61109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:37.580427 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671925e-96cb-4a79-875e-58502aa61109 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:37.653531 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:37.693871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f529a3a-3d85-4d02-a02a-32222023c881 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:37.694952 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f529a3a-3d85-4d02-a02a-32222023c881 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:37.695230 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f529a3a-3d85-4d02-a02a-32222023c881 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:37.695503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f529a3a-3d85-4d02-a02a-32222023c881 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:37.697896 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1f529a3a-3d85-4d02-a02a-32222023c881 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:37.787560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f529a3a-3d85-4d02-a02a-32222023c881 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:37.789728 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f529a3a-3d85-4d02-a02a-32222023c881 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:37.807562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 700/1416] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:38.713378 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c940cb-e5c1-4969-9329-9071e8512bdf None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:38.713945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c940cb-e5c1-4969-9329-9071e8512bdf None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:38.713945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c940cb-e5c1-4969-9329-9071e8512bdf None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:38.715305 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:38.720183 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9f976-fd7e-4153-bad2-c9058bbaa54f None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:38.720855 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9f976-fd7e-4153-bad2-c9058bbaa54f None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:38.720855 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9f976-fd7e-4153-bad2-c9058bbaa54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:38.796716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-33f9f976-fd7e-4153-bad2-c9058bbaa54f None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:38.797843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-33f9f976-fd7e-4153-bad2-c9058bbaa54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:38.824390 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:38.894200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:39.045833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c940cb-e5c1-4969-9329-9071e8512bdf None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:39.047450 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-79c940cb-e5c1-4969-9329-9071e8512bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:39.193554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c940cb-e5c1-4969-9329-9071e8512bdf None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:39.236530 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:39.308761 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7025a3-a32e-4971-8f82-6366c42830e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:39.309283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7025a3-a32e-4971-8f82-6366c42830e9 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:39.309561 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7025a3-a32e-4971-8f82-6366c42830e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:39.309834 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7025a3-a32e-4971-8f82-6366c42830e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:39.311414 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fa7025a3-a32e-4971-8f82-6366c42830e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:39.398989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7025a3-a32e-4971-8f82-6366c42830e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:39.400592 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7025a3-a32e-4971-8f82-6366c42830e9 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:39.419370 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 718/1419] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:40.319315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232f915-3c64-4258-bbc0-aab731f10f96 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:40.319863 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232f915-3c64-4258-bbc0-aab731f10f96 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:40.319863 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232f915-3c64-4258-bbc0-aab731f10f96 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:40.321333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:41:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:40.326425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b5db5-2fb4-44ef-8d74-a893c120dd8b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:40.326873 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b5db5-2fb4-44ef-8d74-a893c120dd8b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:40.326986 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b5db5-2fb4-44ef-8d74-a893c120dd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:40.381130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:40.403424 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0d3b5db5-2fb4-44ef-8d74-a893c120dd8b None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:40.404674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0d3b5db5-2fb4-44ef-8d74-a893c120dd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:40.499819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:40.654819 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232f915-3c64-4258-bbc0-aab731f10f96 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:40.656369 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a232f915-3c64-4258-bbc0-aab731f10f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:40.805383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232f915-3c64-4258-bbc0-aab731f10f96 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:40.817478 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:40.920364 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d07e3-99cc-4b5d-84d9-b3701426e1f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:40.921006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d07e3-99cc-4b5d-84d9-b3701426e1f3 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:40.921287 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d07e3-99cc-4b5d-84d9-b3701426e1f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:40.921563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d07e3-99cc-4b5d-84d9-b3701426e1f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:40.923176 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d45d07e3-99cc-4b5d-84d9-b3701426e1f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:41.013928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d07e3-99cc-4b5d-84d9-b3701426e1f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:41.015521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d07e3-99cc-4b5d-84d9-b3701426e1f3 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:41.034127 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 703/1422] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:41.898548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3c764-ac32-47c4-bef8-835c1ff4561c None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:41.899416 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3c764-ac32-47c4-bef8-835c1ff4561c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:41.899416 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3c764-ac32-47c4-bef8-835c1ff4561c None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:41.901058 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:41.906187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39fc7a-892b-4ba2-8c44-040b36b7f3ce None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:41.906685 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39fc7a-892b-4ba2-8c44-040b36b7f3ce None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:41.906796 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39fc7a-892b-4ba2-8c44-040b36b7f3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:41.928753 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:41.985060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-3e39fc7a-892b-4ba2-8c44-040b36b7f3ce None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:41.986287 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3e39fc7a-892b-4ba2-8c44-040b36b7f3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:42.081504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:42.234790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3c764-ac32-47c4-bef8-835c1ff4561c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:42.236245 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-01b3c764-ac32-47c4-bef8-835c1ff4561c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:42.345396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:42.382065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3c764-ac32-47c4-bef8-835c1ff4561c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:42.494346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9966d-05dc-4301-8386-589b801ae700 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:42.494862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9966d-05dc-4301-8386-589b801ae700 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:42.495138 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9966d-05dc-4301-8386-589b801ae700 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:42.495406 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9966d-05dc-4301-8386-589b801ae700 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:42.497010 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d3f9966d-05dc-4301-8386-589b801ae700 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:42.574387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:42.588074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9966d-05dc-4301-8386-589b801ae700 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:42.589731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9966d-05dc-4301-8386-589b801ae700 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:42.607809 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 721/1425] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:43.487259 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434056b3-8f5a-42bc-8a22-d859caa1f490 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:43.487850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434056b3-8f5a-42bc-8a22-d859caa1f490 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:43.487850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434056b3-8f5a-42bc-8a22-d859caa1f490 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:43.489167 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:43.494081 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cabd6-1ee0-4650-b33d-c3d03c848d32 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:43.494516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cabd6-1ee0-4650-b33d-c3d03c848d32 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:43.494619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cabd6-1ee0-4650-b33d-c3d03c848d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:43.572017 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fb6cabd6-1ee0-4650-b33d-c3d03c848d32 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:43.573271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fb6cabd6-1ee0-4650-b33d-c3d03c848d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:43.669290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:43.825420 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434056b3-8f5a-42bc-8a22-d859caa1f490 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:43.826874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-434056b3-8f5a-42bc-8a22-d859caa1f490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:43.913533 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:43.974671 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434056b3-8f5a-42bc-8a22-d859caa1f490 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:44.088594 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315369a9-f790-42e0-b379-a2978f50c2b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:44.089117 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315369a9-f790-42e0-b379-a2978f50c2b7 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:44.089397 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315369a9-f790-42e0-b379-a2978f50c2b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:44.089669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315369a9-f790-42e0-b379-a2978f50c2b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:44.091279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-315369a9-f790-42e0-b379-a2978f50c2b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:44.140503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:44.183953 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315369a9-f790-42e0-b379-a2978f50c2b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:44.185601 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315369a9-f790-42e0-b379-a2978f50c2b7 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:44.205056 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 706/1428] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:45.057130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30fd42-f494-4ca7-9202-1b2e1899e9c2 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:45.057723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30fd42-f494-4ca7-9202-1b2e1899e9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:45.057723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30fd42-f494-4ca7-9202-1b2e1899e9c2 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:45.059018 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:41:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:45.064108 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491236ba-b97c-4d19-bb9d-411d4c58cc23 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:45.064546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491236ba-b97c-4d19-bb9d-411d4c58cc23 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:45.064682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491236ba-b97c-4d19-bb9d-411d4c58cc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:45.141784 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-491236ba-b97c-4d19-bb9d-411d4c58cc23 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:45.143006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-491236ba-b97c-4d19-bb9d-411d4c58cc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:45.243505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:45.397614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30fd42-f494-4ca7-9202-1b2e1899e9c2 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:45.399141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ec30fd42-f494-4ca7-9202-1b2e1899e9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:45.448204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:45.546558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30fd42-f494-4ca7-9202-1b2e1899e9c2 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:45.675665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d8528-97e5-4b9a-b7f9-2a3a072f49b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:45.676264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d8528-97e5-4b9a-b7f9-2a3a072f49b6 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:45.676573 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d8528-97e5-4b9a-b7f9-2a3a072f49b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:45.676850 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d8528-97e5-4b9a-b7f9-2a3a072f49b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:45.678451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-578d8528-97e5-4b9a-b7f9-2a3a072f49b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:45.685635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:45.774750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d8528-97e5-4b9a-b7f9-2a3a072f49b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:45.776416 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d8528-97e5-4b9a-b7f9-2a3a072f49b6 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:45.795084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 724/1431] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:46.724788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75276c-c92a-4805-9cd5-fa4332f9b956 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:46.725354 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75276c-c92a-4805-9cd5-fa4332f9b956 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:46.725354 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75276c-c92a-4805-9cd5-fa4332f9b956 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:46.726663 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:46.731750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443eac9-c58f-47aa-93f6-14921199513e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:46.732192 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443eac9-c58f-47aa-93f6-14921199513e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:46.732300 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443eac9-c58f-47aa-93f6-14921199513e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:46.810404 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a443eac9-c58f-47aa-93f6-14921199513e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:46.811580 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a443eac9-c58f-47aa-93f6-14921199513e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:46.880472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:46.910690 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:47.061736 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75276c-c92a-4805-9cd5-fa4332f9b956 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:47.063203 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-8a75276c-c92a-4805-9cd5-fa4332f9b956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:47.081697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:47.212365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75276c-c92a-4805-9cd5-fa4332f9b956 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:47.301561 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:47.327417 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c376da4-bada-41d6-a260-9ff984f90efa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:47.327949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c376da4-bada-41d6-a260-9ff984f90efa service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:47.328227 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c376da4-bada-41d6-a260-9ff984f90efa service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:47.328533 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c376da4-bada-41d6-a260-9ff984f90efa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:47.330143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2c376da4-bada-41d6-a260-9ff984f90efa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:47.423218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c376da4-bada-41d6-a260-9ff984f90efa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:47.424925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c376da4-bada-41d6-a260-9ff984f90efa service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:47.442549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 709/1434] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:48.328518 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41428d56-3035-4286-b3c0-e1ad5318d076 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:48.329114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41428d56-3035-4286-b3c0-e1ad5318d076 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:48.329114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41428d56-3035-4286-b3c0-e1ad5318d076 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:48.330425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:48.336091 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad70c3-b840-45ef-96b1-a10effa752eb None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:48.336629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad70c3-b840-45ef-96b1-a10effa752eb None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:48.336754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad70c3-b840-45ef-96b1-a10effa752eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:48.411860 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-f4ad70c3-b840-45ef-96b1-a10effa752eb None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:48.413721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-f4ad70c3-b840-45ef-96b1-a10effa752eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:48.453914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:48.508742 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:48.660840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41428d56-3035-4286-b3c0-e1ad5318d076 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:48.662502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-41428d56-3035-4286-b3c0-e1ad5318d076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:48.810121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41428d56-3035-4286-b3c0-e1ad5318d076 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:48.865291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:48.928646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c63295-1497-4201-8211-85f023014f24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:48.929166 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c63295-1497-4201-8211-85f023014f24 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:48.929438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c63295-1497-4201-8211-85f023014f24 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:48.929701 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c63295-1497-4201-8211-85f023014f24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:48.931302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-58c63295-1497-4201-8211-85f023014f24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:49.020990 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c63295-1497-4201-8211-85f023014f24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:49.022671 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c63295-1497-4201-8211-85f023014f24 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:49.041295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 727/1437] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:49.898862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b63859-d382-4967-ac8b-0b838d0dde27 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:49.899460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b63859-d382-4967-ac8b-0b838d0dde27 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:49.899460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b63859-d382-4967-ac8b-0b838d0dde27 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:49.900819 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:49.906177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8696f-e394-48e2-8508-2dc7afc311e4 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:49.906650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8696f-e394-48e2-8508-2dc7afc311e4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:49.906760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8696f-e394-48e2-8508-2dc7afc311e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:49.981128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a9e8696f-e394-48e2-8508-2dc7afc311e4 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:49.982315 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a9e8696f-e394-48e2-8508-2dc7afc311e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:49.984331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:50.080144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:50.232567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b63859-d382-4967-ac8b-0b838d0dde27 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:50.233900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-77b63859-d382-4967-ac8b-0b838d0dde27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:50.378105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b63859-d382-4967-ac8b-0b838d0dde27 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:50.405634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:50.490965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1b106-acd9-44da-ac30-4062db2d06c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:50.491472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1b106-acd9-44da-ac30-4062db2d06c7 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:50.491747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1b106-acd9-44da-ac30-4062db2d06c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:50.492024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1b106-acd9-44da-ac30-4062db2d06c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:50.493638 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-29c1b106-acd9-44da-ac30-4062db2d06c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:50.583425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1b106-acd9-44da-ac30-4062db2d06c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:50.585051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1b106-acd9-44da-ac30-4062db2d06c7 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:50.602826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 712/1440] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:52.422523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92396c3-cf5d-4e09-bba6-23a6e57d7f55 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:52.423120 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92396c3-cf5d-4e09-bba6-23a6e57d7f55 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:52.423120 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92396c3-cf5d-4e09-bba6-23a6e57d7f55 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:52.424409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:52.430424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed518e2a-d33e-4582-b505-6c028db95888 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:52.430819 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed518e2a-d33e-4582-b505-6c028db95888 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:52.430971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed518e2a-d33e-4582-b505-6c028db95888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:52.471322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:52.496976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-ed518e2a-d33e-4582-b505-6c028db95888 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:52.498186 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ed518e2a-d33e-4582-b505-6c028db95888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:52.596393 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:52.886830 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92396c3-cf5d-4e09-bba6-23a6e57d7f55 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:52.888379 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a92396c3-cf5d-4e09-bba6-23a6e57d7f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:52.994278 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92396c3-cf5d-4e09-bba6-23a6e57d7f55 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:53.026737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:53.072578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f7dfd-78bb-4245-8fea-bd4e32089d14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:53.073212 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f7dfd-78bb-4245-8fea-bd4e32089d14 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:53.073489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f7dfd-78bb-4245-8fea-bd4e32089d14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:53.073756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f7dfd-78bb-4245-8fea-bd4e32089d14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:53.075399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-009f7dfd-78bb-4245-8fea-bd4e32089d14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:53.183345 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f7dfd-78bb-4245-8fea-bd4e32089d14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:53.184834 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f7dfd-78bb-4245-8fea-bd4e32089d14 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:53.204709 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 730/1443] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:55.284732 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee52050-a51e-45e9-82bf-58894fcb2666 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:55.285761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee52050-a51e-45e9-82bf-58894fcb2666 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:55.285761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee52050-a51e-45e9-82bf-58894fcb2666 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:55.286691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:55.292233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03eeca-767b-4f93-9bc6-0cba27f3386c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:55.292705 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03eeca-767b-4f93-9bc6-0cba27f3386c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:55.292922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03eeca-767b-4f93-9bc6-0cba27f3386c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:55.360220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9b03eeca-767b-4f93-9bc6-0cba27f3386c None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:55.361464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9b03eeca-767b-4f93-9bc6-0cba27f3386c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:55.374835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:55.458272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:55.739430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee52050-a51e-45e9-82bf-58894fcb2666 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:55.741093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-dee52050-a51e-45e9-82bf-58894fcb2666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:55.845992 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee52050-a51e-45e9-82bf-58894fcb2666 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:55.858786 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:55.921466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f8b22-56fe-48d5-8ac4-0fcfbccbd4c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:55.922096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f8b22-56fe-48d5-8ac4-0fcfbccbd4c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:55.922360 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f8b22-56fe-48d5-8ac4-0fcfbccbd4c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:55.922634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f8b22-56fe-48d5-8ac4-0fcfbccbd4c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:55.924543 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ab1f8b22-56fe-48d5-8ac4-0fcfbccbd4c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:56.033740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f8b22-56fe-48d5-8ac4-0fcfbccbd4c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:56.035186 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f8b22-56fe-48d5-8ac4-0fcfbccbd4c1 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:56.055900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 715/1446] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:58.413343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d670419-a7ec-4951-8bfc-8d2816945352 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:58.413914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d670419-a7ec-4951-8bfc-8d2816945352 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:58.413914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d670419-a7ec-4951-8bfc-8d2816945352 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:58.415243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:41:58.420564 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8a1b9-167a-445c-baab-3de0153f75b5 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:58.420964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8a1b9-167a-445c-baab-3de0153f75b5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:58.421124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8a1b9-167a-445c-baab-3de0153f75b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:58.468509 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:58.486187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-c4e8a1b9-167a-445c-baab-3de0153f75b5 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:58.487424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c4e8a1b9-167a-445c-baab-3de0153f75b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:58.586766 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:41:58.870833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d670419-a7ec-4951-8bfc-8d2816945352 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:58.872381 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0d670419-a7ec-4951-8bfc-8d2816945352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:58.981637 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d670419-a7ec-4951-8bfc-8d2816945352 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:59.021538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:59.055858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ff755-aed0-4720-a975-1a6729f86e4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:59.056496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ff755-aed0-4720-a975-1a6729f86e4a service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:59.056780 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ff755-aed0-4720-a975-1a6729f86e4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:59.057057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ff755-aed0-4720-a975-1a6729f86e4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:59.058671 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bf2ff755-aed0-4720-a975-1a6729f86e4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:59.167440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ff755-aed0-4720-a975-1a6729f86e4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:59.168828 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ff755-aed0-4720-a975-1a6729f86e4a service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:59.188071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 733/1449] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:01.211978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca79fa6-ddc3-481e-b915-d1f385e20758 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:01.212589 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca79fa6-ddc3-481e-b915-d1f385e20758 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:01.212589 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca79fa6-ddc3-481e-b915-d1f385e20758 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:01.213982 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:42:01.219352 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55548afc-6e0b-49b4-aa87-b1d6a7e668c2 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:01.219757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55548afc-6e0b-49b4-aa87-b1d6a7e668c2 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:01.219904 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55548afc-6e0b-49b4-aa87-b1d6a7e668c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:01.287105 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-55548afc-6e0b-49b4-aa87-b1d6a7e668c2 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:01.288341 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-55548afc-6e0b-49b4-aa87-b1d6a7e668c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:01.308663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:01.385535 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:01.674520 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca79fa6-ddc3-481e-b915-d1f385e20758 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:01.676137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-bca79fa6-ddc3-481e-b915-d1f385e20758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:01.785773 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca79fa6-ddc3-481e-b915-d1f385e20758 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:01.806908 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:01.862438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5211a3-1d27-40fc-bfa3-9d99b0d6f4ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:01.863049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5211a3-1d27-40fc-bfa3-9d99b0d6f4ad service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:01.863328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5211a3-1d27-40fc-bfa3-9d99b0d6f4ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:01.863599 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5211a3-1d27-40fc-bfa3-9d99b0d6f4ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:01.865769 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-5c5211a3-1d27-40fc-bfa3-9d99b0d6f4ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:01.979665 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5211a3-1d27-40fc-bfa3-9d99b0d6f4ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:01.984493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5211a3-1d27-40fc-bfa3-9d99b0d6f4ad service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:01.999713 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 718/1452] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:04.051839 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ea9ac-da7d-47a9-9e70-f23cd29d48ca None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:04.052385 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ea9ac-da7d-47a9-9e70-f23cd29d48ca None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:04.052385 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ea9ac-da7d-47a9-9e70-f23cd29d48ca None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:04.054260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:42:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:04.059591 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ec09d-5a9e-46bf-9104-94d208d6a34e None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:04.060043 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ec09d-5a9e-46bf-9104-94d208d6a34e None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:04.060150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ec09d-5a9e-46bf-9104-94d208d6a34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:04.126395 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:04.129438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-1c6ec09d-5a9e-46bf-9104-94d208d6a34e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:04.130641 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1c6ec09d-5a9e-46bf-9104-94d208d6a34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:04.354225 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:04.638052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ea9ac-da7d-47a9-9e70-f23cd29d48ca None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:04.639599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-280ea9ac-da7d-47a9-9e70-f23cd29d48ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:04.742657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ea9ac-da7d-47a9-9e70-f23cd29d48ca None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:04.792394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:04.817701 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788eed30-05e4-4d76-8328-ab3bd47c677c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:04.818869 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788eed30-05e4-4d76-8328-ab3bd47c677c service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:04.819144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788eed30-05e4-4d76-8328-ab3bd47c677c service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:04.819415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788eed30-05e4-4d76-8328-ab3bd47c677c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:04.821043 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-788eed30-05e4-4d76-8328-ab3bd47c677c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:04.930488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788eed30-05e4-4d76-8328-ab3bd47c677c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:04.931832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788eed30-05e4-4d76-8328-ab3bd47c677c service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:04.951960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 736/1455] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:06.931690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460caec-a706-46e9-ba5f-448c7ca1b24e None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:06.932295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460caec-a706-46e9-ba5f-448c7ca1b24e None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:06.932295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460caec-a706-46e9-ba5f-448c7ca1b24e None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:06.933636 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:06.939102 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1816a1f-d0cd-449d-b638-80a314fa84b5 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:06.939563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1816a1f-d0cd-449d-b638-80a314fa84b5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:06.939674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1816a1f-d0cd-449d-b638-80a314fa84b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:07.006548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e1816a1f-d0cd-449d-b638-80a314fa84b5 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:07.007808 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e1816a1f-d0cd-449d-b638-80a314fa84b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:07.036491 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:07.106440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:07.395437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460caec-a706-46e9-ba5f-448c7ca1b24e None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:07.397709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-4460caec-a706-46e9-ba5f-448c7ca1b24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:07.501305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460caec-a706-46e9-ba5f-448c7ca1b24e None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:07.530753 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:07.577887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bd5f1-532f-45f1-9bed-277098b57c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:07.579004 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bd5f1-532f-45f1-9bed-277098b57c6c service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:07.579004 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bd5f1-532f-45f1-9bed-277098b57c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:07.579004 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bd5f1-532f-45f1-9bed-277098b57c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:07.580644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-cd7bd5f1-532f-45f1-9bed-277098b57c6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:07.691190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bd5f1-532f-45f1-9bed-277098b57c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:07.692571 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bd5f1-532f-45f1-9bed-277098b57c6c service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:07.711015 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 721/1458] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:09.681840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ca146-c9a0-4c0d-9c22-eadc697d964a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:09.682402 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ca146-c9a0-4c0d-9c22-eadc697d964a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:09.682402 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ca146-c9a0-4c0d-9c22-eadc697d964a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:09.683722 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:09.689046 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecd556-36cd-4ee9-a698-595b5adb9874 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:09.689454 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecd556-36cd-4ee9-a698-595b5adb9874 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:09.689627 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecd556-36cd-4ee9-a698-595b5adb9874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:09.758191 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-88ecd556-36cd-4ee9-a698-595b5adb9874 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:09.759393 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-88ecd556-36cd-4ee9-a698-595b5adb9874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:09.767783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:09.855536 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:10.135507 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ca146-c9a0-4c0d-9c22-eadc697d964a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:10.137123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3c2ca146-c9a0-4c0d-9c22-eadc697d964a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:10.242392 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ca146-c9a0-4c0d-9c22-eadc697d964a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:10.298492 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:10.317046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8570389-6799-4da0-8658-7c25cc01afb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:10.317655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8570389-6799-4da0-8658-7c25cc01afb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:10.318509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8570389-6799-4da0-8658-7c25cc01afb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:10.318973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8570389-6799-4da0-8658-7c25cc01afb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:10.321065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d8570389-6799-4da0-8658-7c25cc01afb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:10.430501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8570389-6799-4da0-8658-7c25cc01afb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:10.431903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8570389-6799-4da0-8658-7c25cc01afb9 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:10.450641 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 739/1461] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:12.437281 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48621dde-f685-4189-aa0c-4414aedc0c43 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:12.437825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48621dde-f685-4189-aa0c-4414aedc0c43 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:12.437825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48621dde-f685-4189-aa0c-4414aedc0c43 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:12.439178 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:12.444630 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2604a-05ac-4eeb-b099-9be6c4d55bcd None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:12.445095 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2604a-05ac-4eeb-b099-9be6c4d55bcd None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:12.445207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2604a-05ac-4eeb-b099-9be6c4d55bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:12.638734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-51d2604a-05ac-4eeb-b099-9be6c4d55bcd None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:12.639963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-51d2604a-05ac-4eeb-b099-9be6c4d55bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:12.678363 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:12.735121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:13.024688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48621dde-f685-4189-aa0c-4414aedc0c43 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:13.026236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-48621dde-f685-4189-aa0c-4414aedc0c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:13.133252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48621dde-f685-4189-aa0c-4414aedc0c43 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:13.172427 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:13.207169 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167b6ff-9be4-4edb-971c-951facef4a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:13.207779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167b6ff-9be4-4edb-971c-951facef4a0c service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:13.208831 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167b6ff-9be4-4edb-971c-951facef4a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:13.209049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167b6ff-9be4-4edb-971c-951facef4a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:13.210656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b167b6ff-9be4-4edb-971c-951facef4a0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:13.319845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167b6ff-9be4-4edb-971c-951facef4a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:13.321220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167b6ff-9be4-4edb-971c-951facef4a0c service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:13.339817 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 724/1464] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:15.268804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942e9c5-6aca-4c53-a88e-648afafefc0d None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:15.269387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942e9c5-6aca-4c53-a88e-648afafefc0d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:15.269387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942e9c5-6aca-4c53-a88e-648afafefc0d None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:15.271343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:42:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:15.276879 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d76fac-abe7-4267-8529-54d564fe3fb4 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:15.277367 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d76fac-abe7-4267-8529-54d564fe3fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:15.277477 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d76fac-abe7-4267-8529-54d564fe3fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:15.344635 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-91d76fac-abe7-4267-8529-54d564fe3fb4 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:15.345850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-91d76fac-abe7-4267-8529-54d564fe3fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:15.364470 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:15.443956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:42:15.731088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942e9c5-6aca-4c53-a88e-648afafefc0d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:15.733226 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e942e9c5-6aca-4c53-a88e-648afafefc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:15.840286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942e9c5-6aca-4c53-a88e-648afafefc0d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:15.909036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:15.915752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e47cf3-e159-443f-a294-4f3676db1199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:15.921033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e47cf3-e159-443f-a294-4f3676db1199 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:15.921033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e47cf3-e159-443f-a294-4f3676db1199 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:15.921033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e47cf3-e159-443f-a294-4f3676db1199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:15.921033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-03e47cf3-e159-443f-a294-4f3676db1199 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:16.027280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e47cf3-e159-443f-a294-4f3676db1199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:16.028762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e47cf3-e159-443f-a294-4f3676db1199 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:16.046802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 742/1467] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:18.013586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9d4fc-77d1-4a49-bf6a-36a81bfff062 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:18.014152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9d4fc-77d1-4a49-bf6a-36a81bfff062 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:18.014152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9d4fc-77d1-4a49-bf6a-36a81bfff062 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:18.015522 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:18.021049 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbeca9-bc09-42c8-9349-fbaf5a1ab095 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:18.021505 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbeca9-bc09-42c8-9349-fbaf5a1ab095 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:18.021611 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbeca9-bc09-42c8-9349-fbaf5a1ab095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:18.088259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-04dbeca9-bc09-42c8-9349-fbaf5a1ab095 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:18.089496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-04dbeca9-bc09-42c8-9349-fbaf5a1ab095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:18.137216 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:18.185456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:42:18.469938 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9d4fc-77d1-4a49-bf6a-36a81bfff062 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:18.471486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1ce9d4fc-77d1-4a49-bf6a-36a81bfff062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:18.581040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9d4fc-77d1-4a49-bf6a-36a81bfff062 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:18.627935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:18.654339 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35e1d0-3579-47ba-b461-48078dd7f183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:18.654939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35e1d0-3579-47ba-b461-48078dd7f183 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:18.655216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35e1d0-3579-47ba-b461-48078dd7f183 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:18.655503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35e1d0-3579-47ba-b461-48078dd7f183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:18.657108 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7c35e1d0-3579-47ba-b461-48078dd7f183 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:18.770529 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35e1d0-3579-47ba-b461-48078dd7f183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:18.771879 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35e1d0-3579-47ba-b461-48078dd7f183 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:18.790919 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 727/1470] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:28.183637 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=22 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:28.788617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=66 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:38.428546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4482a6f-2289-41f8-924e-576d33e5508e None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:38.429093 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4482a6f-2289-41f8-924e-576d33e5508e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:38.429093 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4482a6f-2289-41f8-924e-576d33e5508e None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:38.430488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 744/1471] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:38.435794 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d17cc-2fcf-4bb3-9fa1-849e9bb04341 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:38.436240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d17cc-2fcf-4bb3-9fa1-849e9bb04341 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:38.436363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d17cc-2fcf-4bb3-9fa1-849e9bb04341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:38.501352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-6f0d17cc-2fcf-4bb3-9fa1-849e9bb04341 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:38.502568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6f0d17cc-2fcf-4bb3-9fa1-849e9bb04341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:38.602013 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 728/1472] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:38.906163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfeb1d3-ff89-42a3-a13b-4535890cc6e6 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:38.906446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfeb1d3-ff89-42a3-a13b-4535890cc6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:38.906721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfeb1d3-ff89-42a3-a13b-4535890cc6e6 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:38.908041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 745/1473] 10.222.0.70 () {54 vars in 967 bytes} [Wed Aug 24 16:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:38.913178 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2d133-90da-45f8-83f4-3c1cab844905 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:38.913467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2d133-90da-45f8-83f4-3c1cab844905 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:38.913745 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2d133-90da-45f8-83f4-3c1cab844905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:38.988968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-82e2d133-90da-45f8-83f4-3c1cab844905 None None] MFA Rules not processed for user `56418ff737504181a59714aad9f1c375`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:38.990181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-82e2d133-90da-45f8-83f4-3c1cab844905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:39.010994 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:39.066562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 729/1474] 10.222.0.70 () {58 vars in 1064 bytes} [Wed Aug 24 16:42:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:39.074682 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfeb1d3-ff89-42a3-a13b-4535890cc6e6 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:39.076236 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dbfeb1d3-ff89-42a3-a13b-4535890cc6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:39.181764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfeb1d3-ff89-42a3-a13b-4535890cc6e6 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:39.254983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a997a2-0036-4b05-ab00-dc9ec8ced4a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56418ff737504181a59714aad9f1c375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:39.255618 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a997a2-0036-4b05-ab00-dc9ec8ced4a5 service glance] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:39.255895 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a997a2-0036-4b05-ab00-dc9ec8ced4a5 service glance] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:39.256162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a997a2-0036-4b05-ab00-dc9ec8ced4a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:39.257875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-54a997a2-0036-4b05-ab00-dc9ec8ced4a5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:39.311373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:39.365427 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a997a2-0036-4b05-ab00-dc9ec8ced4a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:39.366849 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a997a2-0036-4b05-ab00-dc9ec8ced4a5 service glance] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:39.384938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 746/1475] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:41.398659 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e319d-be1c-4cc1-bc88-962423a550f3 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:41.399222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e319d-be1c-4cc1-bc88-962423a550f3 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:41.399322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e319d-be1c-4cc1-bc88-962423a550f3 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:41.400671 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 730/1476] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:41.406033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4c4f2-71c9-4221-9700-aec8a598cd08 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:41.406493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4c4f2-71c9-4221-9700-aec8a598cd08 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:41.406594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4c4f2-71c9-4221-9700-aec8a598cd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:41.472100 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ddf4c4f2-71c9-4221-9700-aec8a598cd08 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:41.473914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ddf4c4f2-71c9-4221-9700-aec8a598cd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:41.571125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 747/1477] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:41.846152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e319d-be1c-4cc1-bc88-962423a550f3 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:41.848204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7e6e319d-be1c-4cc1-bc88-962423a550f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:41.976091 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e319d-be1c-4cc1-bc88-962423a550f3 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:42.026258 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:42.071352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5717e-e8c2-4050-8b4b-a7af3111b45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:42.071953 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5717e-e8c2-4050-8b4b-a7af3111b45a admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:42.072237 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5717e-e8c2-4050-8b4b-a7af3111b45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:42.072544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5717e-e8c2-4050-8b4b-a7af3111b45a admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:42.073187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5717e-e8c2-4050-8b4b-a7af3111b45a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:42.074915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: WARNING py.warnings [None req-ccc5717e-e8c2-4050-8b4b-a7af3111b45a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:42:42.074915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: warnings.warn(msg) Aug 24 16:42:42.074915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: Aug 24 16:42:42.076052 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5717e-e8c2-4050-8b4b-a7af3111b45a admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:42.095111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 731/1478] 10.222.0.70 () {56 vars in 1108 bytes} [Wed Aug 24 16:42:41 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:42:44.107950 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440da0e-96a5-420c-bd4c-730407bfa12a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:44.108561 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440da0e-96a5-420c-bd4c-730407bfa12a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:44.108561 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440da0e-96a5-420c-bd4c-730407bfa12a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:44.109898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 748/1479] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:44.115758 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8c2af-f651-478b-961f-811d66ef7f95 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:44.116190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8c2af-f651-478b-961f-811d66ef7f95 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:44.116308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8c2af-f651-478b-961f-811d66ef7f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:44.181587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-a5e8c2af-f651-478b-961f-811d66ef7f95 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:44.182804 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a5e8c2af-f651-478b-961f-811d66ef7f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:44.281920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 732/1480] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:44.578402 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440da0e-96a5-420c-bd4c-730407bfa12a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:44.579950 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4440da0e-96a5-420c-bd4c-730407bfa12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:44.590821 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:44.731584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440da0e-96a5-420c-bd4c-730407bfa12a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:44.843049 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ad1c4-2d2c-4b97-9721-34dbbd338f8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:44.843577 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ad1c4-2d2c-4b97-9721-34dbbd338f8c service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:44.843857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ad1c4-2d2c-4b97-9721-34dbbd338f8c service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:44.844129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ad1c4-2d2c-4b97-9721-34dbbd338f8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:44.845755 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4f1ad1c4-2d2c-4b97-9721-34dbbd338f8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:44.850063 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:44.937996 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ad1c4-2d2c-4b97-9721-34dbbd338f8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:44.939620 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ad1c4-2d2c-4b97-9721-34dbbd338f8c service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:44.957413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 749/1481] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:47.093302 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c89ab6-986d-4b42-8841-992317384630 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:47.093918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c89ab6-986d-4b42-8841-992317384630 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:47.093918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c89ab6-986d-4b42-8841-992317384630 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:47.095178 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 733/1482] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:47.100503 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32bde6-4d13-44f7-8d6a-b42b8c5e2602 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:47.100948 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32bde6-4d13-44f7-8d6a-b42b8c5e2602 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:47.101057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32bde6-4d13-44f7-8d6a-b42b8c5e2602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:47.167700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8b32bde6-4d13-44f7-8d6a-b42b8c5e2602 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:47.169015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8b32bde6-4d13-44f7-8d6a-b42b8c5e2602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:47.217206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:47.268456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 750/1483] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:47.555270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c89ab6-986d-4b42-8841-992317384630 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:47.556887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-48c89ab6-986d-4b42-8841-992317384630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:47.581474 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:47.662018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c89ab6-986d-4b42-8841-992317384630 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:47.741465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa708ea-fea4-407b-a437-4f9202d00640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:47.742070 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa708ea-fea4-407b-a437-4f9202d00640 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:47.742349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa708ea-fea4-407b-a437-4f9202d00640 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:47.742624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa708ea-fea4-407b-a437-4f9202d00640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:47.744347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7aa708ea-fea4-407b-a437-4f9202d00640 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:47.819844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:47.854448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa708ea-fea4-407b-a437-4f9202d00640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:47.855909 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa708ea-fea4-407b-a437-4f9202d00640 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:47.874439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 734/1484] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:49.870439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9f19d-e42a-4200-a30b-b786fe251f35 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:49.870985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9f19d-e42a-4200-a30b-b786fe251f35 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:49.870985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9f19d-e42a-4200-a30b-b786fe251f35 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:49.872329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 751/1485] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:49.877899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf600a-6a08-4d12-aa85-47d0ac46d828 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:49.878915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf600a-6a08-4d12-aa85-47d0ac46d828 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:49.879019 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf600a-6a08-4d12-aa85-47d0ac46d828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:49.945208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-2fbf600a-6a08-4d12-aa85-47d0ac46d828 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:49.946407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2fbf600a-6a08-4d12-aa85-47d0ac46d828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:50.045718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 735/1486] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:50.335332 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9f19d-e42a-4200-a30b-b786fe251f35 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:50.336887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-47f9f19d-e42a-4200-a30b-b786fe251f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:50.441687 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9f19d-e42a-4200-a30b-b786fe251f35 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:50.513422 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:50.515270 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38639e32-6aae-4831-955d-7183ece54b54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:50.515915 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38639e32-6aae-4831-955d-7183ece54b54 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:50.516274 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38639e32-6aae-4831-955d-7183ece54b54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:50.516630 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38639e32-6aae-4831-955d-7183ece54b54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:50.518273 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-38639e32-6aae-4831-955d-7183ece54b54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:50.628548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38639e32-6aae-4831-955d-7183ece54b54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:50.629997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38639e32-6aae-4831-955d-7183ece54b54 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:50.647623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 752/1487] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:52.723771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8557a71-a36e-416c-a541-b6c1f3303d54 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:52.724333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8557a71-a36e-416c-a541-b6c1f3303d54 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:52.724333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8557a71-a36e-416c-a541-b6c1f3303d54 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:52.725699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 736/1488] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:52.731276 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723a00c-915d-4d90-b79b-350e0717c28f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:52.731722 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723a00c-915d-4d90-b79b-350e0717c28f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:52.731826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723a00c-915d-4d90-b79b-350e0717c28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:52.803959 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-1723a00c-915d-4d90-b79b-350e0717c28f None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:52.805111 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-1723a00c-915d-4d90-b79b-350e0717c28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:52.856598 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:52.901239 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 753/1489] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:53.191454 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8557a71-a36e-416c-a541-b6c1f3303d54 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:53.193060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a8557a71-a36e-416c-a541-b6c1f3303d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:53.231242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:53.300954 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8557a71-a36e-416c-a541-b6c1f3303d54 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:53.374576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7820b6d-8e0a-43c0-8e3d-98943f4cf394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:53.375179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7820b6d-8e0a-43c0-8e3d-98943f4cf394 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:53.375459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7820b6d-8e0a-43c0-8e3d-98943f4cf394 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:53.375730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7820b6d-8e0a-43c0-8e3d-98943f4cf394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:53.377358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e7820b6d-8e0a-43c0-8e3d-98943f4cf394 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:53.458097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:53.487472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7820b6d-8e0a-43c0-8e3d-98943f4cf394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:53.488944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7820b6d-8e0a-43c0-8e3d-98943f4cf394 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:53.507347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 737/1490] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:55.860693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04629946-0f27-41a6-82f5-328d849c556f None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:55.861254 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04629946-0f27-41a6-82f5-328d849c556f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:55.861352 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04629946-0f27-41a6-82f5-328d849c556f None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:55.863159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 754/1491] 10.222.0.70 () {54 vars in 927 bytes} [Wed Aug 24 16:42:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:55.868897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52a7d8-2ab2-4618-a459-7d4895789f1f None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:55.869374 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52a7d8-2ab2-4618-a459-7d4895789f1f None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:55.869494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52a7d8-2ab2-4618-a459-7d4895789f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:55.936681 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-5e52a7d8-2ab2-4618-a459-7d4895789f1f None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:55.937884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-5e52a7d8-2ab2-4618-a459-7d4895789f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:56.037171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 738/1492] 10.222.0.70 () {58 vars in 1024 bytes} [Wed Aug 24 16:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:56.326916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04629946-0f27-41a6-82f5-328d849c556f None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:56.328499 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-04629946-0f27-41a6-82f5-328d849c556f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:56.433232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04629946-0f27-41a6-82f5-328d849c556f None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:56.505653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3fd0f-eef3-453d-b68c-72c8993e33d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:56.506264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3fd0f-eef3-453d-b68c-72c8993e33d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:56.506546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3fd0f-eef3-453d-b68c-72c8993e33d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:56.506819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3fd0f-eef3-453d-b68c-72c8993e33d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:56.508439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-67d3fd0f-eef3-453d-b68c-72c8993e33d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:56.517499 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:56.617704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3fd0f-eef3-453d-b68c-72c8993e33d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:56.619139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3fd0f-eef3-453d-b68c-72c8993e33d4 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:56.637663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 755/1493] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:43:06.035338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=82 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:43:06.635362 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=50 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:19.666299 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c061e-a432-44cf-9676-42465d40a352 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:19.666885 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c061e-a432-44cf-9676-42465d40a352 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:19.666885 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c061e-a432-44cf-9676-42465d40a352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:19.744912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-1c5c061e-a432-44cf-9676-42465d40a352 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:19.746142 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1c5c061e-a432-44cf-9676-42465d40a352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:19.843920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 739/1494] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:19.852123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa05b2a-1291-476a-a90e-158b99545e18 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:19.852659 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa05b2a-1291-476a-a90e-158b99545e18 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:19.852760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa05b2a-1291-476a-a90e-158b99545e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:19.931112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-dfa05b2a-1291-476a-a90e-158b99545e18 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:19.932308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dfa05b2a-1291-476a-a90e-158b99545e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:20.035173 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 756/1495] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:20.039976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c061e-a432-44cf-9676-42465d40a352 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:20.041259 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1c5c061e-a432-44cf-9676-42465d40a352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:20.053148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c061e-a432-44cf-9676-42465d40a352 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:20.113709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:20.146287 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab56dad-9942-4f17-b7f4-f232cac5a8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:20.146884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab56dad-9942-4f17-b7f4-f232cac5a8fa admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:20.147221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab56dad-9942-4f17-b7f4-f232cac5a8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:20.147502 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab56dad-9942-4f17-b7f4-f232cac5a8fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:20.148341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab56dad-9942-4f17-b7f4-f232cac5a8fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:20.148839 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab56dad-9942-4f17-b7f4-f232cac5a8fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:20.150130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: WARNING py.warnings [None req-6ab56dad-9942-4f17-b7f4-f232cac5a8fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:44:20.150130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: warnings.warn(msg) Aug 24 16:44:20.150130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: Aug 24 16:44:20.151011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab56dad-9942-4f17-b7f4-f232cac5a8fa admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:20.162495 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 740/1496] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:20.171565 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa05b2a-1291-476a-a90e-158b99545e18 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:20.180854 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa05b2a-1291-476a-a90e-158b99545e18 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:20.242098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:20.276864 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a366b-776c-4c3b-b98b-c83eee4e41b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:20.277467 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a366b-776c-4c3b-b98b-c83eee4e41b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:20.277746 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a366b-776c-4c3b-b98b-c83eee4e41b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:20.278014 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a366b-776c-4c3b-b98b-c83eee4e41b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:20.278637 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a366b-776c-4c3b-b98b-c83eee4e41b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:20.280970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: WARNING py.warnings [None req-0b8a366b-776c-4c3b-b98b-c83eee4e41b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:44:20.280970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: warnings.warn(msg) Aug 24 16:44:20.280970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: Aug 24 16:44:20.282156 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a366b-776c-4c3b-b98b-c83eee4e41b4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:20.391033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 757/1497] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:20 2022] POST /identity/v3/projects => generated 379 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:20.395215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab56dad-9942-4f17-b7f4-f232cac5a8fa admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:20.396762 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6ab56dad-9942-4f17-b7f4-f232cac5a8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:20.431115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab56dad-9942-4f17-b7f4-f232cac5a8fa admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:20.526878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5324b-b2be-4807-8972-1fe44bccf65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:20.527486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5324b-b2be-4807-8972-1fe44bccf65e admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:20.527768 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5324b-b2be-4807-8972-1fe44bccf65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:20.528042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5324b-b2be-4807-8972-1fe44bccf65e admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:20.528700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5324b-b2be-4807-8972-1fe44bccf65e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:20.530639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: WARNING py.warnings [None req-1df5324b-b2be-4807-8972-1fe44bccf65e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:44:20.530639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: warnings.warn(msg) Aug 24 16:44:20.530639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: Aug 24 16:44:20.531868 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5324b-b2be-4807-8972-1fe44bccf65e admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:20.663190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 741/1498] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:20 2022] POST /identity/v3/users => generated 290 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:20.667584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a366b-776c-4c3b-b98b-c83eee4e41b4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:20.669181 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0b8a366b-776c-4c3b-b98b-c83eee4e41b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:20.733683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a366b-776c-4c3b-b98b-c83eee4e41b4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:20.816986 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:20.827130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54388208-f9f3-4ee0-965e-b8b2efc28b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:20.831210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54388208-f9f3-4ee0-965e-b8b2efc28b69 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:20.831210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54388208-f9f3-4ee0-965e-b8b2efc28b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:20.831210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54388208-f9f3-4ee0-965e-b8b2efc28b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:20.831210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54388208-f9f3-4ee0-965e-b8b2efc28b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:20.831210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: WARNING py.warnings [None req-54388208-f9f3-4ee0-965e-b8b2efc28b69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:44:20.831210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: warnings.warn(msg) Aug 24 16:44:20.831210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: Aug 24 16:44:20.831823 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54388208-f9f3-4ee0-965e-b8b2efc28b69 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:20.843321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 758/1499] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:20.847448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5324b-b2be-4807-8972-1fe44bccf65e admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:20.849124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1df5324b-b2be-4807-8972-1fe44bccf65e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:20.862215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5324b-b2be-4807-8972-1fe44bccf65e admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:20.889337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:20.959872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75da10f-bcfd-43e3-9aee-10594c98ede6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:20.960390 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75da10f-bcfd-43e3-9aee-10594c98ede6 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:20.960703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75da10f-bcfd-43e3-9aee-10594c98ede6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:20.960969 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75da10f-bcfd-43e3-9aee-10594c98ede6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:20.961598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75da10f-bcfd-43e3-9aee-10594c98ede6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:20.962592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: WARNING py.warnings [None req-b75da10f-bcfd-43e3-9aee-10594c98ede6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:44:20.962592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: warnings.warn(msg) Aug 24 16:44:20.962592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: Aug 24 16:44:20.963469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75da10f-bcfd-43e3-9aee-10594c98ede6 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:20.976842 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 742/1500] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:20.980875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54388208-f9f3-4ee0-965e-b8b2efc28b69 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:20.982392 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-54388208-f9f3-4ee0-965e-b8b2efc28b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:21.015974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54388208-f9f3-4ee0-965e-b8b2efc28b69 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:21.110003 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd56f18-7491-462c-9359-9513b4cbe126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:21.110657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd56f18-7491-462c-9359-9513b4cbe126 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:21.110935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd56f18-7491-462c-9359-9513b4cbe126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:21.111208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd56f18-7491-462c-9359-9513b4cbe126 admin admin] PATH_INFO: `/v3/projects/8dbf69f9281c406faf41980f59f5ab7d/users/e3a89b3bf22e40cfaa26482846c6445c/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:21.133683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd56f18-7491-462c-9359-9513b4cbe126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dbf69f9281c406faf41980f59f5ab7d, user_id=e3a89b3bf22e40cfaa26482846c6445c, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:21.134753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: WARNING py.warnings [None req-bcd56f18-7491-462c-9359-9513b4cbe126 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:44:21.134753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: warnings.warn(msg) Aug 24 16:44:21.134753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: Aug 24 16:44:21.136769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd56f18-7491-462c-9359-9513b4cbe126 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:21.252434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 759/1501] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:20 2022] PUT /identity/v3/projects/8dbf69f9281c406faf41980f59f5ab7d/users/e3a89b3bf22e40cfaa26482846c6445c/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:21.265115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33b360-03a9-4f68-aefc-580e35b42212 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:21.265430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33b360-03a9-4f68-aefc-580e35b42212 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:21.265705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33b360-03a9-4f68-aefc-580e35b42212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:21.323623 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-6f33b360-03a9-4f68-aefc-580e35b42212 None None] MFA Rules not processed for user `e3a89b3bf22e40cfaa26482846c6445c`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:21.324869 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6f33b360-03a9-4f68-aefc-580e35b42212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:21.403365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 743/1502] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:44:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:21.417450 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ca755-08ac-4003-b459-cd3f671494e8 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:21.417742 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ca755-08ac-4003-b459-cd3f671494e8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:21.418026 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ca755-08ac-4003-b459-cd3f671494e8 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:21.419425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 760/1503] 10.222.0.70 () {54 vars in 964 bytes} [Wed Aug 24 16:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:44:21.425127 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873c924-345d-4f63-b700-1f280bc89f0d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:21.425517 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873c924-345d-4f63-b700-1f280bc89f0d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:21.425896 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873c924-345d-4f63-b700-1f280bc89f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:21.458972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:21.505209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-2873c924-345d-4f63-b700-1f280bc89f0d None None] MFA Rules not processed for user `f08140bcbd1e43dda77d4387666f9085`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:21.506377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2873c924-345d-4f63-b700-1f280bc89f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:21.582282 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 744/1504] 10.222.0.70 () {58 vars in 1061 bytes} [Wed Aug 24 16:44:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:21.590056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ca755-08ac-4003-b459-cd3f671494e8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:21.591583 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9d5ca755-08ac-4003-b459-cd3f671494e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:21.642952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:21.698072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ca755-08ac-4003-b459-cd3f671494e8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:21.774880 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd18643-2a01-4038-a769-ce81836a4056 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:21.775396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd18643-2a01-4038-a769-ce81836a4056 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:21.775672 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd18643-2a01-4038-a769-ce81836a4056 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:21.775947 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd18643-2a01-4038-a769-ce81836a4056 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:21.777585 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8dd18643-2a01-4038-a769-ce81836a4056 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:21.884499 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd18643-2a01-4038-a769-ce81836a4056 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:21.886311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd18643-2a01-4038-a769-ce81836a4056 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:21.890039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:21.904817 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 761/1505] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:21.950223 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f724fe-a455-4673-99b2-5393ba3d8cd5 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:21.950672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f724fe-a455-4673-99b2-5393ba3d8cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:21.951078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f724fe-a455-4673-99b2-5393ba3d8cd5 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:21.952924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 745/1506] 10.222.0.70 () {52 vars in 830 bytes} [Wed Aug 24 16:44:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:44:21.956802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd18643-2a01-4038-a769-ce81836a4056 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:21.958031 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8dd18643-2a01-4038-a769-ce81836a4056 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:22.056472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd18643-2a01-4038-a769-ce81836a4056 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:22.149534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df8cc9-9598-42b8-953d-d310ab51cec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:22.150161 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df8cc9-9598-42b8-953d-d310ab51cec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:22.150440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df8cc9-9598-42b8-953d-d310ab51cec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:22.150731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df8cc9-9598-42b8-953d-d310ab51cec8 admin admin] PATH_INFO: `/v3/users/e3a89b3bf22e40cfaa26482846c6445c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:22.163235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df8cc9-9598-42b8-953d-d310ab51cec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3a89b3bf22e40cfaa26482846c6445c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:22.164251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: WARNING py.warnings [None req-06df8cc9-9598-42b8-953d-d310ab51cec8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:44:22.164251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: warnings.warn(msg) Aug 24 16:44:22.164251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: Aug 24 16:44:22.165544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df8cc9-9598-42b8-953d-d310ab51cec8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:22.176577 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:22.375744 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-06df8cc9-9598-42b8-953d-d310ab51cec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3a89b3bf22e40cfaa26482846c6445c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:22.376770 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-06df8cc9-9598-42b8-953d-d310ab51cec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3a89b3bf22e40cfaa26482846c6445c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:22.376770 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-06df8cc9-9598-42b8-953d-d310ab51cec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3a89b3bf22e40cfaa26482846c6445c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:22.466939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-06df8cc9-9598-42b8-953d-d310ab51cec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3a89b3bf22e40cfaa26482846c6445c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:22.482307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-06df8cc9-9598-42b8-953d-d310ab51cec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3a89b3bf22e40cfaa26482846c6445c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:22.512502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 762/1507] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:21 2022] DELETE /identity/v3/users/e3a89b3bf22e40cfaa26482846c6445c => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:22.517557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07514e3d-4677-4493-86e7-e3f1e9a78d12 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:22.517844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07514e3d-4677-4493-86e7-e3f1e9a78d12 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:22.519957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07514e3d-4677-4493-86e7-e3f1e9a78d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:22.575825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-07514e3d-4677-4493-86e7-e3f1e9a78d12 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:22.579137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-07514e3d-4677-4493-86e7-e3f1e9a78d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:22.657260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:22.669246 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 746/1508] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:44:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:22.679551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df8cc9-9598-42b8-953d-d310ab51cec8 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:22.680830 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-06df8cc9-9598-42b8-953d-d310ab51cec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:22.784159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df8cc9-9598-42b8-953d-d310ab51cec8 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:22.837410 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:22.857025 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3528a13-f939-4675-9748-8c3776f72fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:22.857683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3528a13-f939-4675-9748-8c3776f72fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:22.857959 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3528a13-f939-4675-9748-8c3776f72fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:22.858226 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3528a13-f939-4675-9748-8c3776f72fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:22.859864 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c3528a13-f939-4675-9748-8c3776f72fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:22.971289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3528a13-f939-4675-9748-8c3776f72fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:22.972663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3528a13-f939-4675-9748-8c3776f72fc3 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:22.990908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 763/1509] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:23.499499 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07514e3d-4677-4493-86e7-e3f1e9a78d12 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:23.501031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-07514e3d-4677-4493-86e7-e3f1e9a78d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:23.512975 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07514e3d-4677-4493-86e7-e3f1e9a78d12 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:23.619258 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb17496-f5a2-409b-9ba9-e6a579d79539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:23.619897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb17496-f5a2-409b-9ba9-e6a579d79539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:23.620176 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb17496-f5a2-409b-9ba9-e6a579d79539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:23.620478 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb17496-f5a2-409b-9ba9-e6a579d79539 admin admin] PATH_INFO: `/v3/projects/8dbf69f9281c406faf41980f59f5ab7d` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:23.631192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb17496-f5a2-409b-9ba9-e6a579d79539 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dbf69f9281c406faf41980f59f5ab7d)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:23.632204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: WARNING py.warnings [None req-6bb17496-f5a2-409b-9ba9-e6a579d79539 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:44:23.632204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: warnings.warn(msg) Aug 24 16:44:23.632204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: Aug 24 16:44:23.633456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb17496-f5a2-409b-9ba9-e6a579d79539 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:23.729069 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6bb17496-f5a2-409b-9ba9-e6a579d79539 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dbf69f9281c406faf41980f59f5ab7d', 'request_id': 'req-6bb17496-f5a2-409b-9ba9-e6a579d79539'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:23.743289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6bb17496-f5a2-409b-9ba9-e6a579d79539 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dbf69f9281c406faf41980f59f5ab7d', 'request_id': 'req-6bb17496-f5a2-409b-9ba9-e6a579d79539'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:23.750220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6bb17496-f5a2-409b-9ba9-e6a579d79539 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dbf69f9281c406faf41980f59f5ab7d', 'request_id': 'req-6bb17496-f5a2-409b-9ba9-e6a579d79539'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:23.862064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6bb17496-f5a2-409b-9ba9-e6a579d79539 admin admin] The token cache is being invalidate because project 8dbf69f9281c406faf41980f59f5ab7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:23.862842 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6bb17496-f5a2-409b-9ba9-e6a579d79539 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:23.863279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6bb17496-f5a2-409b-9ba9-e6a579d79539 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:23.864746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 747/1510] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:23 2022] DELETE /identity/v3/projects/8dbf69f9281c406faf41980f59f5ab7d => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:32.988631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=62 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:33.859994 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:55.647666 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc2cbc-4f7b-4cf8-a511-31e6889c39a8 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:55.648266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc2cbc-4f7b-4cf8-a511-31e6889c39a8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:55.648266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc2cbc-4f7b-4cf8-a511-31e6889c39a8 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:55.649604 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:45:55.654560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa803464-dea3-4535-957c-60140559beda None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:55.655040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa803464-dea3-4535-957c-60140559beda None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:55.655150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa803464-dea3-4535-957c-60140559beda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:55.735596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-fa803464-dea3-4535-957c-60140559beda None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:55.736823 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-fa803464-dea3-4535-957c-60140559beda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:55.846327 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:56.001978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc2cbc-4f7b-4cf8-a511-31e6889c39a8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:56.004098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5fdc2cbc-4f7b-4cf8-a511-31e6889c39a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:56.125032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc2cbc-4f7b-4cf8-a511-31e6889c39a8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:56.130860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7f26c-83c8-4100-8482-390307ed9eff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:56.131492 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7f26c-83c8-4100-8482-390307ed9eff service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:56.131777 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7f26c-83c8-4100-8482-390307ed9eff service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:56.132054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7f26c-83c8-4100-8482-390307ed9eff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:56.141415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7f26c-83c8-4100-8482-390307ed9eff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:56.143004 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7f26c-83c8-4100-8482-390307ed9eff service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:56.152637 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 765/1513] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:56.986306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43745e05-93aa-4923-a435-a103ce212870 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:56.986875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43745e05-93aa-4923-a435-a103ce212870 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:56.986875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43745e05-93aa-4923-a435-a103ce212870 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:56.988209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:45:56.993200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce760a2a-4d6a-475f-8767-f7ec09502bce None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:56.994601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce760a2a-4d6a-475f-8767-f7ec09502bce None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:56.994885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce760a2a-4d6a-475f-8767-f7ec09502bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:57.014264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ce760a2a-4d6a-475f-8767-f7ec09502bce None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:57.015493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ce760a2a-4d6a-475f-8767-f7ec09502bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:57.023656 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:57.176850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43745e05-93aa-4923-a435-a103ce212870 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:57.191602 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43745e05-93aa-4923-a435-a103ce212870 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:57.197171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77631d8f-ebf9-4eca-8d8b-e7c130bd3206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:57.197781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77631d8f-ebf9-4eca-8d8b-e7c130bd3206 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:57.198064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77631d8f-ebf9-4eca-8d8b-e7c130bd3206 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:57.198340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77631d8f-ebf9-4eca-8d8b-e7c130bd3206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:57.207450 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77631d8f-ebf9-4eca-8d8b-e7c130bd3206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:57.209084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77631d8f-ebf9-4eca-8d8b-e7c130bd3206 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:57.218551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 750/1516] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:59.281066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421466d-0f48-4b9d-8099-95f0957bd891 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:59.281633 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421466d-0f48-4b9d-8099-95f0957bd891 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:59.281633 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421466d-0f48-4b9d-8099-95f0957bd891 None None] PATH_INFO: `/v3` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:59.282904 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 767/1517] 10.222.0.60 () {54 vars in 925 bytes} [Wed Aug 24 16:45:59 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:45:59.289478 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a9501-6ace-4c94-b68f-544430c39695 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:59.289903 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a9501-6ace-4c94-b68f-544430c39695 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:59.290042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a9501-6ace-4c94-b68f-544430c39695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:59.349433 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-b52a9501-6ace-4c94-b68f-544430c39695 None None] MFA Rules not processed for user `fc4c43c85a5d49979ba803fdb1041c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:59.350648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b52a9501-6ace-4c94-b68f-544430c39695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:59.367142 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:59.423426 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 751/1518] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Aug 24 16:45:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:03.786127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd8544-4235-4d10-ae39-ceeb93c6cfa0 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:03.786646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd8544-4235-4d10-ae39-ceeb93c6cfa0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:03.786646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd8544-4235-4d10-ae39-ceeb93c6cfa0 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:03.787996 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 768/1519] 10.222.0.70 () {54 vars in 961 bytes} [Wed Aug 24 16:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:46:03.793318 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a05e3-9344-4592-a3c1-2f9e52a6d6b2 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:03.793782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a05e3-9344-4592-a3c1-2f9e52a6d6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:03.793866 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a05e3-9344-4592-a3c1-2f9e52a6d6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:03.813521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-345a05e3-9344-4592-a3c1-2f9e52a6d6b2 None None] MFA Rules not processed for user `fc4c43c85a5d49979ba803fdb1041c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:03.814813 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-345a05e3-9344-4592-a3c1-2f9e52a6d6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:03.823479 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 752/1520] 10.222.0.70 () {58 vars in 1058 bytes} [Wed Aug 24 16:46:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:03.831317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd8544-4235-4d10-ae39-ceeb93c6cfa0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:03.845345 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd8544-4235-4d10-ae39-ceeb93c6cfa0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:03.850332 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48bb1cc-71bd-4289-9d9b-765a603f6e2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:03.850929 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48bb1cc-71bd-4289-9d9b-765a603f6e2e service placement] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:03.851226 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48bb1cc-71bd-4289-9d9b-765a603f6e2e service placement] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:03.851503 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48bb1cc-71bd-4289-9d9b-765a603f6e2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:03.860645 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48bb1cc-71bd-4289-9d9b-765a603f6e2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:03.862232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48bb1cc-71bd-4289-9d9b-765a603f6e2e service placement] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:03.872180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 769/1521] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:08.434252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a65a5-ad1e-4ac7-a16c-9f1f9627cfe0 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:08.434871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a65a5-ad1e-4ac7-a16c-9f1f9627cfe0 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:08.434871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a65a5-ad1e-4ac7-a16c-9f1f9627cfe0 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:08.436308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:46:08.441878 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755792b-5b66-4f7e-8595-2f3917fc7a99 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:08.442317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755792b-5b66-4f7e-8595-2f3917fc7a99 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:08.442579 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755792b-5b66-4f7e-8595-2f3917fc7a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:08.462206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a755792b-5b66-4f7e-8595-2f3917fc7a99 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:08.494337 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a755792b-5b66-4f7e-8595-2f3917fc7a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:08.494337 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:08.626924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a65a5-ad1e-4ac7-a16c-9f1f9627cfe0 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:08.643027 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a65a5-ad1e-4ac7-a16c-9f1f9627cfe0 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:08.648328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095450b2-82ab-40ae-bfb5-7016c1c7dd86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:08.648949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095450b2-82ab-40ae-bfb5-7016c1c7dd86 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:08.649231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095450b2-82ab-40ae-bfb5-7016c1c7dd86 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:08.649506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095450b2-82ab-40ae-bfb5-7016c1c7dd86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:08.658218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095450b2-82ab-40ae-bfb5-7016c1c7dd86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:08.659803 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095450b2-82ab-40ae-bfb5-7016c1c7dd86 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:08.669236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 754/1524] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:09.518126 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9adb07-1a2e-481a-94b4-6a1b5443ebc1 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:09.518722 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9adb07-1a2e-481a-94b4-6a1b5443ebc1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:09.519133 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9adb07-1a2e-481a-94b4-6a1b5443ebc1 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:09.521755 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:46:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:46:09.528223 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3afecc-db60-4866-a6d0-5db2f02e1105 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:09.528772 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3afecc-db60-4866-a6d0-5db2f02e1105 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:09.529247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3afecc-db60-4866-a6d0-5db2f02e1105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:09.551418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-7f3afecc-db60-4866-a6d0-5db2f02e1105 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:09.552835 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7f3afecc-db60-4866-a6d0-5db2f02e1105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:09.562573 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:09.718545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9adb07-1a2e-481a-94b4-6a1b5443ebc1 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:09.733475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9adb07-1a2e-481a-94b4-6a1b5443ebc1 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:09.738985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea1e06-6fbf-4042-a593-9fd4be627d24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:09.739576 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea1e06-6fbf-4042-a593-9fd4be627d24 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:09.739860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea1e06-6fbf-4042-a593-9fd4be627d24 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:09.740139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea1e06-6fbf-4042-a593-9fd4be627d24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:09.748525 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea1e06-6fbf-4042-a593-9fd4be627d24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:09.750046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea1e06-6fbf-4042-a593-9fd4be627d24 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:09.759320 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 772/1527] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:13.531464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cf2fe-2bbd-480e-a6cc-3fcae55f900f None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:13.532068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cf2fe-2bbd-480e-a6cc-3fcae55f900f None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:13.532068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cf2fe-2bbd-480e-a6cc-3fcae55f900f None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:13.533468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:46:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:46:13.540457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b378dc-10b6-4a15-9ddd-239688b635fd None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:13.540944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b378dc-10b6-4a15-9ddd-239688b635fd None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:13.541115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b378dc-10b6-4a15-9ddd-239688b635fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:13.562221 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c2b378dc-10b6-4a15-9ddd-239688b635fd None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:13.563433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c2b378dc-10b6-4a15-9ddd-239688b635fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:13.571442 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:13.857271 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000cf2fe-2bbd-480e-a6cc-3fcae55f900f None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:13.859396 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-000cf2fe-2bbd-480e-a6cc-3fcae55f900f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:13.939161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000cf2fe-2bbd-480e-a6cc-3fcae55f900f None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:13.942748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633a4dd-b8ed-44e0-8064-9e050d7b7491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:13.943350 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633a4dd-b8ed-44e0-8064-9e050d7b7491 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:13.943625 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633a4dd-b8ed-44e0-8064-9e050d7b7491 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:13.943895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633a4dd-b8ed-44e0-8064-9e050d7b7491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:13.953343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633a4dd-b8ed-44e0-8064-9e050d7b7491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:13.954757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633a4dd-b8ed-44e0-8064-9e050d7b7491 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:13.964057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 757/1530] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:15.874945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5332b04-6cfa-47d3-934d-527a6212e066 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:15.875476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5332b04-6cfa-47d3-934d-527a6212e066 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:15.875586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5332b04-6cfa-47d3-934d-527a6212e066 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:15.877038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:46:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:46:15.882544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3e9af-b2f5-4b47-ad3d-53230c648c9a None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:15.883046 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3e9af-b2f5-4b47-ad3d-53230c648c9a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:15.883157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3e9af-b2f5-4b47-ad3d-53230c648c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:15.903528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-90f3e9af-b2f5-4b47-ad3d-53230c648c9a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:15.904807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-90f3e9af-b2f5-4b47-ad3d-53230c648c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:15.912907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:16.198545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5332b04-6cfa-47d3-934d-527a6212e066 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:16.210315 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5332b04-6cfa-47d3-934d-527a6212e066 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:16.213862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc524d9-aa31-4cda-b13d-471fd8de96ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:16.214461 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc524d9-aa31-4cda-b13d-471fd8de96ee service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:16.214741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc524d9-aa31-4cda-b13d-471fd8de96ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:16.215015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc524d9-aa31-4cda-b13d-471fd8de96ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:16.224785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc524d9-aa31-4cda-b13d-471fd8de96ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:16.226304 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc524d9-aa31-4cda-b13d-471fd8de96ee service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:16.235748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 775/1533] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:19.840742 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b2346-c98d-47c4-abd5-705c6a9c4c9a None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:19.841310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b2346-c98d-47c4-abd5-705c6a9c4c9a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:19.841310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b2346-c98d-47c4-abd5-705c6a9c4c9a None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:19.842581 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:46:19.847717 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5be4a-309f-471e-9472-74fd9aadae02 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:19.848176 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5be4a-309f-471e-9472-74fd9aadae02 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:19.848284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5be4a-309f-471e-9472-74fd9aadae02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:19.867786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0cf5be4a-309f-471e-9472-74fd9aadae02 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:19.868942 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0cf5be4a-309f-471e-9472-74fd9aadae02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:19.877658 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:46:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:20.442862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b2346-c98d-47c4-abd5-705c6a9c4c9a None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:20.444576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-566b2346-c98d-47c4-abd5-705c6a9c4c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:20.512335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:20.525690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b2346-c98d-47c4-abd5-705c6a9c4c9a None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:20.533110 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c59020-e098-4921-be8a-d0b99b96e0e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:20.533110 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c59020-e098-4921-be8a-d0b99b96e0e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:20.533110 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c59020-e098-4921-be8a-d0b99b96e0e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:20.533110 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c59020-e098-4921-be8a-d0b99b96e0e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:20.540726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c59020-e098-4921-be8a-d0b99b96e0e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:20.542198 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c59020-e098-4921-be8a-d0b99b96e0e1 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:20.552876 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 760/1536] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:22.478175 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78be8c-12bc-4775-868b-836333c31716 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:22.478746 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78be8c-12bc-4775-868b-836333c31716 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:22.478868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78be8c-12bc-4775-868b-836333c31716 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:22.480172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:46:22.485543 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c2df3-6800-47f9-9c74-935dd6e6b234 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:22.485957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c2df3-6800-47f9-9c74-935dd6e6b234 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:22.486102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c2df3-6800-47f9-9c74-935dd6e6b234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:22.506360 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-b89c2df3-6800-47f9-9c74-935dd6e6b234 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:22.507565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b89c2df3-6800-47f9-9c74-935dd6e6b234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:22.516694 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:46:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:23.072589 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f78be8c-12bc-4775-868b-836333c31716 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:23.074295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9f78be8c-12bc-4775-868b-836333c31716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:23.081957 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=84 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:23.087637 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f78be8c-12bc-4775-868b-836333c31716 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:23.091169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeedc110-bd7c-4cce-8679-cb2dd0d9f82a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:23.097898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedc110-bd7c-4cce-8679-cb2dd0d9f82a service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:23.097898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedc110-bd7c-4cce-8679-cb2dd0d9f82a service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:23.097898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedc110-bd7c-4cce-8679-cb2dd0d9f82a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:23.101022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeedc110-bd7c-4cce-8679-cb2dd0d9f82a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:23.102435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeedc110-bd7c-4cce-8679-cb2dd0d9f82a service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:23.112626 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 778/1539] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:32.501392 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=11 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:33.108802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:49.781339 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ee02e-2b5c-44c5-9af6-d4279ca1d25e None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:49.781962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ee02e-2b5c-44c5-9af6-d4279ca1d25e None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:49.781962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ee02e-2b5c-44c5-9af6-d4279ca1d25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:49.807190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-dd5ee02e-2b5c-44c5-9af6-d4279ca1d25e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:49.808797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-dd5ee02e-2b5c-44c5-9af6-d4279ca1d25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:49.820459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 762/1540] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:49.829293 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c10881-9e65-4686-9b0e-05030ea39f16 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:49.830225 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c10881-9e65-4686-9b0e-05030ea39f16 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:49.830703 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c10881-9e65-4686-9b0e-05030ea39f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:49.856898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c8c10881-9e65-4686-9b0e-05030ea39f16 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:49.858681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c8c10881-9e65-4686-9b0e-05030ea39f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:49.872010 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 779/1541] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:47:49.876784 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ee02e-2b5c-44c5-9af6-d4279ca1d25e None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:49.890182 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ee02e-2b5c-44c5-9af6-d4279ca1d25e None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:49.897259 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941ead8-8d74-4124-b162-bed922ed3b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:49.898230 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941ead8-8d74-4124-b162-bed922ed3b59 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:49.898710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941ead8-8d74-4124-b162-bed922ed3b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:49.899186 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941ead8-8d74-4124-b162-bed922ed3b59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:49.900085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941ead8-8d74-4124-b162-bed922ed3b59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:47:49.900622 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941ead8-8d74-4124-b162-bed922ed3b59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:49.902806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941ead8-8d74-4124-b162-bed922ed3b59 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:49.923604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 763/1542] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:47:49.932302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c10881-9e65-4686-9b0e-05030ea39f16 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:49.946290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c10881-9e65-4686-9b0e-05030ea39f16 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:49.957392 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4a76f-c52a-4bdf-9162-086cdef7fe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:49.958514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4a76f-c52a-4bdf-9162-086cdef7fe8e admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:49.958995 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4a76f-c52a-4bdf-9162-086cdef7fe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:49.960258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4a76f-c52a-4bdf-9162-086cdef7fe8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:49.961814 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4a76f-c52a-4bdf-9162-086cdef7fe8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:49.964081 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4a76f-c52a-4bdf-9162-086cdef7fe8e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:50.127952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 780/1543] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:47:49 2022] POST /identity/v3/projects => generated 371 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:50.132071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941ead8-8d74-4124-b162-bed922ed3b59 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:50.146051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941ead8-8d74-4124-b162-bed922ed3b59 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:50.150949 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a71fe-be5e-4470-8068-2b3059f0876e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.151283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a71fe-be5e-4470-8068-2b3059f0876e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.151536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a71fe-be5e-4470-8068-2b3059f0876e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.191406 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3c2a71fe-be5e-4470-8068-2b3059f0876e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:50.192673 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3c2a71fe-be5e-4470-8068-2b3059f0876e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:50.302569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27f5a8-a10e-42a4-961d-e0606488171b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:50.308371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27f5a8-a10e-42a4-961d-e0606488171b admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.308683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27f5a8-a10e-42a4-961d-e0606488171b admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.308964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27f5a8-a10e-42a4-961d-e0606488171b admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.309630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27f5a8-a10e-42a4-961d-e0606488171b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:50.316189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27f5a8-a10e-42a4-961d-e0606488171b admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:50.326014 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 781/1544] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:50.328347 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be3c6f-1bed-4f89-ae25-ec951c418555 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.328661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be3c6f-1bed-4f89-ae25-ec951c418555 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.328942 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be3c6f-1bed-4f89-ae25-ec951c418555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.359534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-76be3c6f-1bed-4f89-ae25-ec951c418555 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:50.360824 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-76be3c6f-1bed-4f89-ae25-ec951c418555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:50.371146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 782/1545] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:50.372711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777e19a-d89d-4da9-a15b-f8c7398c01ad None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.373010 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777e19a-d89d-4da9-a15b-f8c7398c01ad None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.374193 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777e19a-d89d-4da9-a15b-f8c7398c01ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.397336 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-6777e19a-d89d-4da9-a15b-f8c7398c01ad None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:50.398523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6777e19a-d89d-4da9-a15b-f8c7398c01ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:50.407813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 783/1546] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:50.409582 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8c0ae-4938-4c66-ab8b-cf9cdeebcb2d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.410003 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8c0ae-4938-4c66-ab8b-cf9cdeebcb2d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.410286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8c0ae-4938-4c66-ab8b-cf9cdeebcb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.433657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-d0a8c0ae-4938-4c66-ab8b-cf9cdeebcb2d None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:50.434875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d0a8c0ae-4938-4c66-ab8b-cf9cdeebcb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:50.443838 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 764/1547] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/users => generated 286 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:47:50.445435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57a952-3191-4e28-ba77-76835a96810f None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.445931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57a952-3191-4e28-ba77-76835a96810f None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.446042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57a952-3191-4e28-ba77-76835a96810f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.459175 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 784/1548] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:50.461042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddc573-8771-4038-9183-e3c83eb0facc None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.461336 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddc573-8771-4038-9183-e3c83eb0facc None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.461611 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddc573-8771-4038-9183-e3c83eb0facc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.470676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-ce57a952-3191-4e28-ba77-76835a96810f None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:50.471998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ce57a952-3191-4e28-ba77-76835a96810f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:50.480631 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 765/1549] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:50.481442 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57a952-3191-4e28-ba77-76835a96810f None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:50.484718 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0bddc573-8771-4038-9183-e3c83eb0facc None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:50.486075 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0bddc573-8771-4038-9183-e3c83eb0facc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:50.492283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57a952-3191-4e28-ba77-76835a96810f None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:50.495555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 785/1550] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:50.496999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bfd9f-261e-4958-84e4-3b66a0760dbf None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.497288 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bfd9f-261e-4958-84e4-3b66a0760dbf None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.497560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bfd9f-261e-4958-84e4-3b66a0760dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.498071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0d336-87a8-4a75-ae79-881c0bd9c197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:50.498688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0d336-87a8-4a75-ae79-881c0bd9c197 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.498971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0d336-87a8-4a75-ae79-881c0bd9c197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.499269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0d336-87a8-4a75-ae79-881c0bd9c197 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.499924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0d336-87a8-4a75-ae79-881c0bd9c197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:50.501557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0d336-87a8-4a75-ae79-881c0bd9c197 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:50.515218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 766/1551] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:50.515218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a11b1b-5791-4136-a2ed-fc3bb8713883 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.515218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a11b1b-5791-4136-a2ed-fc3bb8713883 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.515218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a11b1b-5791-4136-a2ed-fc3bb8713883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.517543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9b9bfd9f-261e-4958-84e4-3b66a0760dbf None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:50.518768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9b9bfd9f-261e-4958-84e4-3b66a0760dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:50.526734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 786/1552] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:50.527585 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bfd9f-261e-4958-84e4-3b66a0760dbf None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:50.533629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-68a11b1b-5791-4136-a2ed-fc3bb8713883 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:50.534795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-68a11b1b-5791-4136-a2ed-fc3bb8713883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:50.537740 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bfd9f-261e-4958-84e4-3b66a0760dbf None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:50.544624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a96441-7715-4865-ad2a-900e59aa2783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:50.545263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a96441-7715-4865-ad2a-900e59aa2783 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.545543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a96441-7715-4865-ad2a-900e59aa2783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.545816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a96441-7715-4865-ad2a-900e59aa2783 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.546352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 767/1553] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:50.546459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a96441-7715-4865-ad2a-900e59aa2783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:47:50.546758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a96441-7715-4865-ad2a-900e59aa2783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:50.547571 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08f04a-29ae-4e7c-8403-f3a63aa883c3 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.547830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08f04a-29ae-4e7c-8403-f3a63aa883c3 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.547936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: WARNING py.warnings [None req-a5a96441-7715-4865-ad2a-900e59aa2783 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:47:50.547936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: warnings.warn(msg) Aug 24 16:47:50.547936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: Aug 24 16:47:50.548166 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08f04a-29ae-4e7c-8403-f3a63aa883c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.548759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a96441-7715-4865-ad2a-900e59aa2783 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:50.572836 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 787/1554] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:47:50.572836 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a96441-7715-4865-ad2a-900e59aa2783 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:50.573156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-8e08f04a-29ae-4e7c-8403-f3a63aa883c3 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:50.573156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-8e08f04a-29ae-4e7c-8403-f3a63aa883c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:50.577554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 768/1555] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:50.578515 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a96441-7715-4865-ad2a-900e59aa2783 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:50.578741 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0a955-73ce-4e52-9073-5b3bca0c37cb None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.579025 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0a955-73ce-4e52-9073-5b3bca0c37cb None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.579297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0a955-73ce-4e52-9073-5b3bca0c37cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.588899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918807b-7c5e-4726-8f5b-dcab95fb428a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:50.589513 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918807b-7c5e-4726-8f5b-dcab95fb428a admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.589812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918807b-7c5e-4726-8f5b-dcab95fb428a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.590083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918807b-7c5e-4726-8f5b-dcab95fb428a admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.590784 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918807b-7c5e-4726-8f5b-dcab95fb428a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:47:50.591088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918807b-7c5e-4726-8f5b-dcab95fb428a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:50.592954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918807b-7c5e-4726-8f5b-dcab95fb428a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:50.599686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-7ee0a955-73ce-4e52-9073-5b3bca0c37cb None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:50.600945 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7ee0a955-73ce-4e52-9073-5b3bca0c37cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:50.609217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 788/1556] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:47:50.609217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918807b-7c5e-4726-8f5b-dcab95fb428a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:50.609468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 769/1557] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:50.611070 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0a955-73ce-4e52-9073-5b3bca0c37cb None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:50.617694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918807b-7c5e-4726-8f5b-dcab95fb428a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:50.622006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0a955-73ce-4e52-9073-5b3bca0c37cb None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:50.623576 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bf163-955d-428f-9989-754d4e6b2030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:50.624283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bf163-955d-428f-9989-754d4e6b2030 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.624591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bf163-955d-428f-9989-754d4e6b2030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.624870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bf163-955d-428f-9989-754d4e6b2030 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.625513 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bf163-955d-428f-9989-754d4e6b2030 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:47:50.625819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bf163-955d-428f-9989-754d4e6b2030 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:50.626679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f22c2-cdc6-4330-b37d-cbb8af9c48c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:50.627269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f22c2-cdc6-4330-b37d-cbb8af9c48c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.627408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bf163-955d-428f-9989-754d4e6b2030 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:50.627554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f22c2-cdc6-4330-b37d-cbb8af9c48c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.627825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f22c2-cdc6-4330-b37d-cbb8af9c48c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.628495 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f22c2-cdc6-4330-b37d-cbb8af9c48c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:50.630130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f22c2-cdc6-4330-b37d-cbb8af9c48c2 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:50.646977 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 770/1558] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:50.647175 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 789/1559] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:47:50.647295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6cf01-1a81-4c4d-8435-b26abc8ac778 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.647435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae828866-7ea2-4a9e-8a15-bdff60db35fe None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.647534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6cf01-1a81-4c4d-8435-b26abc8ac778 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.647636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae828866-7ea2-4a9e-8a15-bdff60db35fe None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.647740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6cf01-1a81-4c4d-8435-b26abc8ac778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.647850 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae828866-7ea2-4a9e-8a15-bdff60db35fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.660869 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-f5c6cf01-1a81-4c4d-8435-b26abc8ac778 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:50.662075 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-f5c6cf01-1a81-4c4d-8435-b26abc8ac778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:50.663256 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ae828866-7ea2-4a9e-8a15-bdff60db35fe None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:50.664457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ae828866-7ea2-4a9e-8a15-bdff60db35fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:50.670467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 771/1560] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:50.671351 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6cf01-1a81-4c4d-8435-b26abc8ac778 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:50.676558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 790/1561] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:50.677763 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c753fc-83e6-44a3-a801-c7715e524ceb None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.678048 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c753fc-83e6-44a3-a801-c7715e524ceb None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.678323 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c753fc-83e6-44a3-a801-c7715e524ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.684601 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6cf01-1a81-4c4d-8435-b26abc8ac778 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:50.689161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea61a91-3626-41d7-b0dd-171cfb4708c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:50.689753 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea61a91-3626-41d7-b0dd-171cfb4708c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.690031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea61a91-3626-41d7-b0dd-171cfb4708c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.690304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea61a91-3626-41d7-b0dd-171cfb4708c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.690922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea61a91-3626-41d7-b0dd-171cfb4708c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:50.691964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: WARNING py.warnings [None req-fea61a91-3626-41d7-b0dd-171cfb4708c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:47:50.691964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: warnings.warn(msg) Aug 24 16:47:50.691964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: Aug 24 16:47:50.693190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea61a91-3626-41d7-b0dd-171cfb4708c9 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:50.697644 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-55c753fc-83e6-44a3-a801-c7715e524ceb None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:50.698833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-55c753fc-83e6-44a3-a801-c7715e524ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:50.706801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 791/1562] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:50.707375 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c753fc-83e6-44a3-a801-c7715e524ceb None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:50.716650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c753fc-83e6-44a3-a801-c7715e524ceb None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:50.721219 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160dae6-df8c-4d63-95a2-c0671352d80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:50.721794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160dae6-df8c-4d63-95a2-c0671352d80e admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.722074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160dae6-df8c-4d63-95a2-c0671352d80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.722344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160dae6-df8c-4d63-95a2-c0671352d80e admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.722960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160dae6-df8c-4d63-95a2-c0671352d80e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:50.724881 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160dae6-df8c-4d63-95a2-c0671352d80e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:50.774527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 772/1563] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/projects => generated 391 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:50.776343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fce3b6-00be-4953-8cf6-cf654733593d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.776660 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fce3b6-00be-4953-8cf6-cf654733593d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.776930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fce3b6-00be-4953-8cf6-cf654733593d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.795565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-f2fce3b6-00be-4953-8cf6-cf654733593d None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:50.796808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-f2fce3b6-00be-4953-8cf6-cf654733593d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:50.832151 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 792/1564] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:50.833617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160dae6-df8c-4d63-95a2-c0671352d80e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:50.843750 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:50.853259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:50.856126 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 773/1565] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:50.856770 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fce3b6-00be-4953-8cf6-cf654733593d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:50.866040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fce3b6-00be-4953-8cf6-cf654733593d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:50.907771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160dae6-df8c-4d63-95a2-c0671352d80e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:50.912632 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82749b45-7744-4fe3-b56f-37f5fd4b5676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:50.920689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82749b45-7744-4fe3-b56f-37f5fd4b5676 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.920689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82749b45-7744-4fe3-b56f-37f5fd4b5676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.920689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82749b45-7744-4fe3-b56f-37f5fd4b5676 admin admin] PATH_INFO: `/v3/projects/48651184e30d4c2480d534554351a7df/users/af29bb9fd4974e64908c696620065dde/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.926524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168c42c-6810-46a3-93a4-5fa5d5b59d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:50.927156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168c42c-6810-46a3-93a4-5fa5d5b59d52 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:50.927598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168c42c-6810-46a3-93a4-5fa5d5b59d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:50.927881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168c42c-6810-46a3-93a4-5fa5d5b59d52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:50.928545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168c42c-6810-46a3-93a4-5fa5d5b59d52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:50.928650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82749b45-7744-4fe3-b56f-37f5fd4b5676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48651184e30d4c2480d534554351a7df, user_id=af29bb9fd4974e64908c696620065dde, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:50.930458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168c42c-6810-46a3-93a4-5fa5d5b59d52 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:50.931740 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82749b45-7744-4fe3-b56f-37f5fd4b5676 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.045327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 793/1566] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:47:50 2022] PUT /identity/v3/projects/48651184e30d4c2480d534554351a7df/users/af29bb9fd4974e64908c696620065dde/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:51.046219 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82749b45-7744-4fe3-b56f-37f5fd4b5676 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.055964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 774/1567] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:47:50 2022] POST /identity/v3/projects => generated 389 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:51.057646 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168c42c-6810-46a3-93a4-5fa5d5b59d52 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.115396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82749b45-7744-4fe3-b56f-37f5fd4b5676 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.118961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168c42c-6810-46a3-93a4-5fa5d5b59d52 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.120389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400377c0-28ec-4e8b-9fe0-baf6a39cde21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.121020 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400377c0-28ec-4e8b-9fe0-baf6a39cde21 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.121296 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400377c0-28ec-4e8b-9fe0-baf6a39cde21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.121562 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400377c0-28ec-4e8b-9fe0-baf6a39cde21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.122304 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400377c0-28ec-4e8b-9fe0-baf6a39cde21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:47:51.122609 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400377c0-28ec-4e8b-9fe0-baf6a39cde21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.123888 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c5d61-b07f-40b0-bbba-c99d3c701007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.124185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400377c0-28ec-4e8b-9fe0-baf6a39cde21 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.124647 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c5d61-b07f-40b0-bbba-c99d3c701007 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.124930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c5d61-b07f-40b0-bbba-c99d3c701007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.125206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c5d61-b07f-40b0-bbba-c99d3c701007 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.125852 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c5d61-b07f-40b0-bbba-c99d3c701007 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:47:51.126158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c5d61-b07f-40b0-bbba-c99d3c701007 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.127792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c5d61-b07f-40b0-bbba-c99d3c701007 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.143199 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 794/1568] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:47:51.143199 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400377c0-28ec-4e8b-9fe0-baf6a39cde21 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.143456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 775/1569] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:47:51.143456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c5d61-b07f-40b0-bbba-c99d3c701007 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.152886 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400377c0-28ec-4e8b-9fe0-baf6a39cde21 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.157172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c5d61-b07f-40b0-bbba-c99d3c701007 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.157445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7d55d-3ebf-4390-84f7-40cf67ded478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.158104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7d55d-3ebf-4390-84f7-40cf67ded478 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.158323 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7d55d-3ebf-4390-84f7-40cf67ded478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.158589 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7d55d-3ebf-4390-84f7-40cf67ded478 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.159225 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7d55d-3ebf-4390-84f7-40cf67ded478 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:47:51.159525 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7d55d-3ebf-4390-84f7-40cf67ded478 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.161144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7d55d-3ebf-4390-84f7-40cf67ded478 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.161956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421cc84-d001-49b0-b94b-1dd60142ed4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.162559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421cc84-d001-49b0-b94b-1dd60142ed4e admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.162834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421cc84-d001-49b0-b94b-1dd60142ed4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.163112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421cc84-d001-49b0-b94b-1dd60142ed4e admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.163740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421cc84-d001-49b0-b94b-1dd60142ed4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.166341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421cc84-d001-49b0-b94b-1dd60142ed4e admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.173701 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 795/1570] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:47:51.174555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7d55d-3ebf-4390-84f7-40cf67ded478 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.183880 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7d55d-3ebf-4390-84f7-40cf67ded478 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.188830 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f177e3-ca24-4c1d-85f2-c0edfea1e05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.189423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f177e3-ca24-4c1d-85f2-c0edfea1e05b admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.189705 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f177e3-ca24-4c1d-85f2-c0edfea1e05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.189980 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f177e3-ca24-4c1d-85f2-c0edfea1e05b admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.190739 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f177e3-ca24-4c1d-85f2-c0edfea1e05b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.191748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: WARNING py.warnings [None req-a0f177e3-ca24-4c1d-85f2-c0edfea1e05b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:47:51.191748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: warnings.warn(msg) Aug 24 16:47:51.191748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: Aug 24 16:47:51.193034 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f177e3-ca24-4c1d-85f2-c0edfea1e05b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.320658 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 776/1571] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:47:51 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:51.321730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421cc84-d001-49b0-b94b-1dd60142ed4e admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.331796 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421cc84-d001-49b0-b94b-1dd60142ed4e admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.336491 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f338a0-3a0c-40b8-84d9-d4cef30170af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.337074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f338a0-3a0c-40b8-84d9-d4cef30170af admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.337357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f338a0-3a0c-40b8-84d9-d4cef30170af admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.338269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f338a0-3a0c-40b8-84d9-d4cef30170af admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.339044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f338a0-3a0c-40b8-84d9-d4cef30170af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:47:51.339355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f338a0-3a0c-40b8-84d9-d4cef30170af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.341014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f338a0-3a0c-40b8-84d9-d4cef30170af admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.353796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 796/1572] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:47:51 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:51.353796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5e926-e74c-4870-9103-99a4fe6d7105 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.353796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5e926-e74c-4870-9103-99a4fe6d7105 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.353796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5e926-e74c-4870-9103-99a4fe6d7105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.354240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 777/1573] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:47:51.361068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f338a0-3a0c-40b8-84d9-d4cef30170af admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.370790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f338a0-3a0c-40b8-84d9-d4cef30170af admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.375503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774d23e-e641-413c-8fe8-d79a0323f373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.376084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774d23e-e641-413c-8fe8-d79a0323f373 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.376357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774d23e-e641-413c-8fe8-d79a0323f373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.376654 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774d23e-e641-413c-8fe8-d79a0323f373 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.377267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774d23e-e641-413c-8fe8-d79a0323f373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.379149 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774d23e-e641-413c-8fe8-d79a0323f373 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.380248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a7d5e926-e74c-4870-9103-99a4fe6d7105 None None] MFA Rules not processed for user `af29bb9fd4974e64908c696620065dde`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:51.381564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a7d5e926-e74c-4870-9103-99a4fe6d7105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:51.428369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:51.444537 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 797/1574] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:51.445326 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5e926-e74c-4870-9103-99a4fe6d7105 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.455654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5e926-e74c-4870-9103-99a4fe6d7105 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.460194 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f7234-6571-43d2-9d0b-9057b99dd8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.460814 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f7234-6571-43d2-9d0b-9057b99dd8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.461098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f7234-6571-43d2-9d0b-9057b99dd8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.461371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f7234-6571-43d2-9d0b-9057b99dd8f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.461992 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f7234-6571-43d2-9d0b-9057b99dd8f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.463226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 778/1575] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:47:51 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:51.464286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774d23e-e641-413c-8fe8-d79a0323f373 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.466234 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f7234-6571-43d2-9d0b-9057b99dd8f2 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.473910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774d23e-e641-413c-8fe8-d79a0323f373 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.478431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e5f2f-89b5-4129-bac0-5ecc2d1067b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.479017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e5f2f-89b5-4129-bac0-5ecc2d1067b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.479293 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e5f2f-89b5-4129-bac0-5ecc2d1067b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.479561 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e5f2f-89b5-4129-bac0-5ecc2d1067b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.480184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e5f2f-89b5-4129-bac0-5ecc2d1067b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.482246 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e5f2f-89b5-4129-bac0-5ecc2d1067b0 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.567228 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 798/1576] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:47:51 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:51.567631 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 779/1577] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:47:51 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:51.568639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e5f2f-89b5-4129-bac0-5ecc2d1067b0 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.571432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f7234-6571-43d2-9d0b-9057b99dd8f2 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.574872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:51.632150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e5f2f-89b5-4129-bac0-5ecc2d1067b0 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.632385 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f7234-6571-43d2-9d0b-9057b99dd8f2 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.637505 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f4f61-89a8-4e14-9a44-e367610ccf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.637773 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c8bf8-9500-46c6-9490-6a74ec53eeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.638232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f4f61-89a8-4e14-9a44-e367610ccf5c admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.638374 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c8bf8-9500-46c6-9490-6a74ec53eeda admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.638521 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f4f61-89a8-4e14-9a44-e367610ccf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.638656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c8bf8-9500-46c6-9490-6a74ec53eeda admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.638799 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f4f61-89a8-4e14-9a44-e367610ccf5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.638932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c8bf8-9500-46c6-9490-6a74ec53eeda admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.639429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f4f61-89a8-4e14-9a44-e367610ccf5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.639573 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c8bf8-9500-46c6-9490-6a74ec53eeda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.641187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c8bf8-9500-46c6-9490-6a74ec53eeda admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.644140 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f4f61-89a8-4e14-9a44-e367610ccf5c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.651405 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 780/1578] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:51.652212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c8bf8-9500-46c6-9490-6a74ec53eeda admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.661514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c8bf8-9500-46c6-9490-6a74ec53eeda admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.666881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc73666-7725-4134-a58b-e0a39e98f2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.667476 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc73666-7725-4134-a58b-e0a39e98f2fc admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.668545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc73666-7725-4134-a58b-e0a39e98f2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.668545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc73666-7725-4134-a58b-e0a39e98f2fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.668745 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc73666-7725-4134-a58b-e0a39e98f2fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.670323 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc73666-7725-4134-a58b-e0a39e98f2fc admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.680237 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 781/1579] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:51.681034 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc73666-7725-4134-a58b-e0a39e98f2fc admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.690936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc73666-7725-4134-a58b-e0a39e98f2fc admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.695787 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f336c36-f585-45ba-9bf1-b20e15fadaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.697555 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f336c36-f585-45ba-9bf1-b20e15fadaa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.697555 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f336c36-f585-45ba-9bf1-b20e15fadaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.697723 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f336c36-f585-45ba-9bf1-b20e15fadaa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.698237 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f336c36-f585-45ba-9bf1-b20e15fadaa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.700087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f336c36-f585-45ba-9bf1-b20e15fadaa9 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.724435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 799/1580] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:47:51 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:51.724435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f4f61-89a8-4e14-9a44-e367610ccf5c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.780907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 782/1581] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:47:51 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:51.781881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f336c36-f585-45ba-9bf1-b20e15fadaa9 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.785662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f4f61-89a8-4e14-9a44-e367610ccf5c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.841622 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ada61d-426b-41c4-8078-a1c3cc8c9887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.842311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ada61d-426b-41c4-8078-a1c3cc8c9887 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.842667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ada61d-426b-41c4-8078-a1c3cc8c9887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.842944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ada61d-426b-41c4-8078-a1c3cc8c9887 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.843570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ada61d-426b-41c4-8078-a1c3cc8c9887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.845508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ada61d-426b-41c4-8078-a1c3cc8c9887 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.846005 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f336c36-f585-45ba-9bf1-b20e15fadaa9 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.850611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6a653-b95d-4f5e-be99-1ef4930f1368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.851047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:51.851230 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6a653-b95d-4f5e-be99-1ef4930f1368 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.851494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6a653-b95d-4f5e-be99-1ef4930f1368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.851763 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6a653-b95d-4f5e-be99-1ef4930f1368 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.852422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6a653-b95d-4f5e-be99-1ef4930f1368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.854053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6a653-b95d-4f5e-be99-1ef4930f1368 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.864377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 783/1582] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:51.865218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6a653-b95d-4f5e-be99-1ef4930f1368 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.875504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6a653-b95d-4f5e-be99-1ef4930f1368 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:51.880094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb12e68-ecaf-4f17-a78f-b276aff5090a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:51.880707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb12e68-ecaf-4f17-a78f-b276aff5090a admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:51.880988 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb12e68-ecaf-4f17-a78f-b276aff5090a admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:51.886576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb12e68-ecaf-4f17-a78f-b276aff5090a admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:51.886576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb12e68-ecaf-4f17-a78f-b276aff5090a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:51.886576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb12e68-ecaf-4f17-a78f-b276aff5090a admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:51.941325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 800/1583] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:47:51 2022] POST /identity/v3/projects => generated 371 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:51.942164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ada61d-426b-41c4-8078-a1c3cc8c9887 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.972554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 784/1584] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:47:51 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:51.973399 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb12e68-ecaf-4f17-a78f-b276aff5090a admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:51.982923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb12e68-ecaf-4f17-a78f-b276aff5090a admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.005691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ada61d-426b-41c4-8078-a1c3cc8c9887 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.010962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dea94-fdea-4c0c-b62f-0bf186365998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.011576 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dea94-fdea-4c0c-b62f-0bf186365998 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.011893 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dea94-fdea-4c0c-b62f-0bf186365998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.012169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dea94-fdea-4c0c-b62f-0bf186365998 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.012834 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73dea94-fdea-4c0c-b62f-0bf186365998 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.014745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73dea94-fdea-4c0c-b62f-0bf186365998 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.026613 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:52.041480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb6b84-2eb5-4df0-aab6-73e40b0fd6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.042089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb6b84-2eb5-4df0-aab6-73e40b0fd6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.042375 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb6b84-2eb5-4df0-aab6-73e40b0fd6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.042648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb6b84-2eb5-4df0-aab6-73e40b0fd6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.043290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb6b84-2eb5-4df0-aab6-73e40b0fd6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.044910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb6b84-2eb5-4df0-aab6-73e40b0fd6b7 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.054453 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 785/1585] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:52.055236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb6b84-2eb5-4df0-aab6-73e40b0fd6b7 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.064873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb6b84-2eb5-4df0-aab6-73e40b0fd6b7 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.069399 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e1ded-6af2-41ca-b877-073cb30b8acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.069949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e1ded-6af2-41ca-b877-073cb30b8acb admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.070232 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e1ded-6af2-41ca-b877-073cb30b8acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.070505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e1ded-6af2-41ca-b877-073cb30b8acb admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.071108 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e1ded-6af2-41ca-b877-073cb30b8acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.072677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e1ded-6af2-41ca-b877-073cb30b8acb admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.082862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 786/1586] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:52.086880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e1ded-6af2-41ca-b877-073cb30b8acb admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.092776 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e1ded-6af2-41ca-b877-073cb30b8acb admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.097342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee85696-c333-4c8c-aede-b1ed7cd47376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.097894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee85696-c333-4c8c-aede-b1ed7cd47376 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.098175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee85696-c333-4c8c-aede-b1ed7cd47376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.098448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee85696-c333-4c8c-aede-b1ed7cd47376 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.099042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee85696-c333-4c8c-aede-b1ed7cd47376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.100902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee85696-c333-4c8c-aede-b1ed7cd47376 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.114719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 801/1587] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:47:51 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:52.116092 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dea94-fdea-4c0c-b62f-0bf186365998 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.126295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dea94-fdea-4c0c-b62f-0bf186365998 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.131009 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062ab76-d96e-4ccf-a73f-c69dc634e611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.131582 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062ab76-d96e-4ccf-a73f-c69dc634e611 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.131867 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062ab76-d96e-4ccf-a73f-c69dc634e611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.132142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062ab76-d96e-4ccf-a73f-c69dc634e611 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.132802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062ab76-d96e-4ccf-a73f-c69dc634e611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.134696 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062ab76-d96e-4ccf-a73f-c69dc634e611 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.237524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 787/1588] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:47:52 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:52.237524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee85696-c333-4c8c-aede-b1ed7cd47376 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.247703 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 802/1589] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:47:52 2022] POST /identity/v3/users => generated 286 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:52.252142 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee85696-c333-4c8c-aede-b1ed7cd47376 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.252259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062ab76-d96e-4ccf-a73f-c69dc634e611 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.254358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e04725-7f29-4996-b8ed-6c761c35a3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.254965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e04725-7f29-4996-b8ed-6c761c35a3c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.255246 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e04725-7f29-4996-b8ed-6c761c35a3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.255520 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e04725-7f29-4996-b8ed-6c761c35a3c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.256199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e04725-7f29-4996-b8ed-6c761c35a3c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.257350 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062ab76-d96e-4ccf-a73f-c69dc634e611 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.258098 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e04725-7f29-4996-b8ed-6c761c35a3c3 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.262192 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bed6fc-bdac-4e87-8e97-584817698437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.262826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bed6fc-bdac-4e87-8e97-584817698437 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.263108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bed6fc-bdac-4e87-8e97-584817698437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.263382 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bed6fc-bdac-4e87-8e97-584817698437 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.264005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bed6fc-bdac-4e87-8e97-584817698437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.265943 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bed6fc-bdac-4e87-8e97-584817698437 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.271642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 788/1590] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:52.271642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e04725-7f29-4996-b8ed-6c761c35a3c3 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.279310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e04725-7f29-4996-b8ed-6c761c35a3c3 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.283892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4958973-0dc0-4d90-9454-be9296f1524f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.284501 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4958973-0dc0-4d90-9454-be9296f1524f admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.284781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4958973-0dc0-4d90-9454-be9296f1524f admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.285054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4958973-0dc0-4d90-9454-be9296f1524f admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.285684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4958973-0dc0-4d90-9454-be9296f1524f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.287262 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4958973-0dc0-4d90-9454-be9296f1524f admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.298314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 789/1591] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:52.299220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4958973-0dc0-4d90-9454-be9296f1524f admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.310931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4958973-0dc0-4d90-9454-be9296f1524f admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.317269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c5f7c-8eb9-4a06-a0f9-629c52ad4de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.318114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c5f7c-8eb9-4a06-a0f9-629c52ad4de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.318518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c5f7c-8eb9-4a06-a0f9-629c52ad4de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.318918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c5f7c-8eb9-4a06-a0f9-629c52ad4de5 admin admin] PATH_INFO: `/v3/projects/42f5635b43e74a96a56d56cccbc111b5/users/45913fc3d9ba42ea9e971db25e656c61/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.335458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c5f7c-8eb9-4a06-a0f9-629c52ad4de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f5635b43e74a96a56d56cccbc111b5, user_id=45913fc3d9ba42ea9e971db25e656c61, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.336508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: WARNING py.warnings [None req-a31c5f7c-8eb9-4a06-a0f9-629c52ad4de5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:47:52.336508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: warnings.warn(msg) Aug 24 16:47:52.336508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: Aug 24 16:47:52.338417 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c5f7c-8eb9-4a06-a0f9-629c52ad4de5 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.374754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 803/1592] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:47:52 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:52.375637 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bed6fc-bdac-4e87-8e97-584817698437 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.385017 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bed6fc-bdac-4e87-8e97-584817698437 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.389595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9f5fc-da89-49b0-8afc-5ed2b9c76b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.390213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9f5fc-da89-49b0-8afc-5ed2b9c76b16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.390495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9f5fc-da89-49b0-8afc-5ed2b9c76b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.390766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9f5fc-da89-49b0-8afc-5ed2b9c76b16 admin admin] PATH_INFO: `/v3/projects/3b5a3a82d90e4811a3037a8cda927913/users/e2257adc34324452a325bb7786d02515/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.405988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9f5fc-da89-49b0-8afc-5ed2b9c76b16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b5a3a82d90e4811a3037a8cda927913, user_id=e2257adc34324452a325bb7786d02515, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.408849 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9f5fc-da89-49b0-8afc-5ed2b9c76b16 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.428553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 790/1593] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:47:52 2022] PUT /identity/v3/projects/42f5635b43e74a96a56d56cccbc111b5/users/45913fc3d9ba42ea9e971db25e656c61/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:52.429386 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c5f7c-8eb9-4a06-a0f9-629c52ad4de5 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.468858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 804/1594] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:47:52 2022] PUT /identity/v3/projects/3b5a3a82d90e4811a3037a8cda927913/users/e2257adc34324452a325bb7786d02515/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:52.469768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9f5fc-da89-49b0-8afc-5ed2b9c76b16 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.491611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c5f7c-8eb9-4a06-a0f9-629c52ad4de5 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.530184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9f5fc-da89-49b0-8afc-5ed2b9c76b16 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.534907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e1b71-3984-4dff-93db-d32ac3367d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.535511 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e1b71-3984-4dff-93db-d32ac3367d40 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.535789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e1b71-3984-4dff-93db-d32ac3367d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.536065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e1b71-3984-4dff-93db-d32ac3367d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.536740 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e1b71-3984-4dff-93db-d32ac3367d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.538384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e1b71-3984-4dff-93db-d32ac3367d40 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.548611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0fc89-ec3d-4197-97d2-3c068695c477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.549225 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0fc89-ec3d-4197-97d2-3c068695c477 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.549438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 805/1595] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:52.552572 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0fc89-ec3d-4197-97d2-3c068695c477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.552572 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0fc89-ec3d-4197-97d2-3c068695c477 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.552773 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e1b71-3984-4dff-93db-d32ac3367d40 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.552882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0fc89-ec3d-4197-97d2-3c068695c477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.552882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0fc89-ec3d-4197-97d2-3c068695c477 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.561988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e1b71-3984-4dff-93db-d32ac3367d40 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.562862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 791/1596] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:52.563630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0fc89-ec3d-4197-97d2-3c068695c477 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.568648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fc62c-01c7-45f8-a449-3d07bc567647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.569460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fc62c-01c7-45f8-a449-3d07bc567647 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.569742 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fc62c-01c7-45f8-a449-3d07bc567647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.570572 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fc62c-01c7-45f8-a449-3d07bc567647 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.571342 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fc62c-01c7-45f8-a449-3d07bc567647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.572875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fc62c-01c7-45f8-a449-3d07bc567647 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.573624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0fc89-ec3d-4197-97d2-3c068695c477 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.578328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cf084-2575-4301-9204-48665a93d551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.578960 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cf084-2575-4301-9204-48665a93d551 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.579240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cf084-2575-4301-9204-48665a93d551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.579515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cf084-2575-4301-9204-48665a93d551 admin admin] PATH_INFO: `/v3/projects/525245253a2d49c48ef009a4d6c3cd51/users/7a930257a6584b04bae278aa83940264/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.586445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 806/1597] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:52.586622 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:52.586723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fc62c-01c7-45f8-a449-3d07bc567647 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.593415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:52.595953 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cf084-2575-4301-9204-48665a93d551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=525245253a2d49c48ef009a4d6c3cd51, user_id=7a930257a6584b04bae278aa83940264, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.598652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cf084-2575-4301-9204-48665a93d551 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.604514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fc62c-01c7-45f8-a449-3d07bc567647 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.618892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c370533-520b-4885-97fd-e4726cf83cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.622851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c370533-520b-4885-97fd-e4726cf83cfe admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.622851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c370533-520b-4885-97fd-e4726cf83cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.622851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c370533-520b-4885-97fd-e4726cf83cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.622851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c370533-520b-4885-97fd-e4726cf83cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.622851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c370533-520b-4885-97fd-e4726cf83cfe admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.633913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 807/1598] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:52.635022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c370533-520b-4885-97fd-e4726cf83cfe admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.645189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c370533-520b-4885-97fd-e4726cf83cfe admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.649705 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b05bdd-6280-4151-bd02-86091fe66ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.651106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b05bdd-6280-4151-bd02-86091fe66ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.651106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b05bdd-6280-4151-bd02-86091fe66ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.651106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b05bdd-6280-4151-bd02-86091fe66ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.651928 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b05bdd-6280-4151-bd02-86091fe66ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.653527 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b05bdd-6280-4151-bd02-86091fe66ba7 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.665823 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 808/1599] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:52.667764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11b53a-ed28-494d-933a-4b8b49a30812 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.668052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11b53a-ed28-494d-933a-4b8b49a30812 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.668330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11b53a-ed28-494d-933a-4b8b49a30812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.685933 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-af11b53a-ed28-494d-933a-4b8b49a30812 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:52.687265 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-af11b53a-ed28-494d-933a-4b8b49a30812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:52.694907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 809/1600] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:52.696180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc831d91-4c3d-4820-b5f8-2ede482d0756 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.696505 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc831d91-4c3d-4820-b5f8-2ede482d0756 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.696786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc831d91-4c3d-4820-b5f8-2ede482d0756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.709001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 792/1601] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:47:52 2022] PUT /identity/v3/projects/525245253a2d49c48ef009a4d6c3cd51/users/7a930257a6584b04bae278aa83940264/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:52.710075 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cf084-2575-4301-9204-48665a93d551 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.715870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-dc831d91-4c3d-4820-b5f8-2ede482d0756 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:52.717123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dc831d91-4c3d-4820-b5f8-2ede482d0756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:52.772519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cf084-2575-4301-9204-48665a93d551 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.775328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 810/1602] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:52.776219 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc831d91-4c3d-4820-b5f8-2ede482d0756 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.782054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c9c42-520c-40fc-ad98-17b2b36c3e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.782682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c9c42-520c-40fc-ad98-17b2b36c3e08 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.782968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c9c42-520c-40fc-ad98-17b2b36c3e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.783242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c9c42-520c-40fc-ad98-17b2b36c3e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.783981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c9c42-520c-40fc-ad98-17b2b36c3e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.785540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c9c42-520c-40fc-ad98-17b2b36c3e08 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.786244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc831d91-4c3d-4820-b5f8-2ede482d0756 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.791629 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42701e27-a676-44b8-9b64-419842ff8815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.792258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42701e27-a676-44b8-9b64-419842ff8815 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.792569 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42701e27-a676-44b8-9b64-419842ff8815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.792846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42701e27-a676-44b8-9b64-419842ff8815 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.793489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42701e27-a676-44b8-9b64-419842ff8815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.795066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42701e27-a676-44b8-9b64-419842ff8815 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.799137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 793/1603] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:52.799137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c9c42-520c-40fc-ad98-17b2b36c3e08 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.805371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 811/1604] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:52.806039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42701e27-a676-44b8-9b64-419842ff8815 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.806448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c9c42-520c-40fc-ad98-17b2b36c3e08 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.811039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319764c-367b-48d8-bff4-4e1ca238f231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.811622 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319764c-367b-48d8-bff4-4e1ca238f231 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.811907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319764c-367b-48d8-bff4-4e1ca238f231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.812176 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319764c-367b-48d8-bff4-4e1ca238f231 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.812844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319764c-367b-48d8-bff4-4e1ca238f231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.814419 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319764c-367b-48d8-bff4-4e1ca238f231 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.815512 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42701e27-a676-44b8-9b64-419842ff8815 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.819984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793f5d8-3b22-497b-8e74-7edb86ac219f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.820644 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793f5d8-3b22-497b-8e74-7edb86ac219f admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.820943 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793f5d8-3b22-497b-8e74-7edb86ac219f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.821197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793f5d8-3b22-497b-8e74-7edb86ac219f admin admin] PATH_INFO: `/v3/projects/260e43af05a741139d93472a06effdd5/users/f7bee141038c437eb57f3aa0e97558f1/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.827858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 794/1605] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:52.827858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319764c-367b-48d8-bff4-4e1ca238f231 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.834800 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319764c-367b-48d8-bff4-4e1ca238f231 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:52.837565 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793f5d8-3b22-497b-8e74-7edb86ac219f admin admin] RBAC: Authorizing `identity:create_grant(project_id=260e43af05a741139d93472a06effdd5, user_id=f7bee141038c437eb57f3aa0e97558f1, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.839949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f5c74-f29e-4998-8ed1-018ac108e0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:52.840607 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f5c74-f29e-4998-8ed1-018ac108e0c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.840883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793f5d8-3b22-497b-8e74-7edb86ac219f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.841004 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f5c74-f29e-4998-8ed1-018ac108e0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.841222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f5c74-f29e-4998-8ed1-018ac108e0c4 admin admin] PATH_INFO: `/v3/projects/3f12cf6c0d3f4ef8946e98d2e03f8194/users/3c01a452981647908bb4e1ace42d3bff/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.855534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f5c74-f29e-4998-8ed1-018ac108e0c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f12cf6c0d3f4ef8946e98d2e03f8194, user_id=3c01a452981647908bb4e1ace42d3bff, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:52.858201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f5c74-f29e-4998-8ed1-018ac108e0c4 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:52.898752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 812/1606] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:47:52 2022] PUT /identity/v3/projects/260e43af05a741139d93472a06effdd5/users/f7bee141038c437eb57f3aa0e97558f1/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:52.899667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793f5d8-3b22-497b-8e74-7edb86ac219f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:52.925276 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 795/1607] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:47:52 2022] PUT /identity/v3/projects/3f12cf6c0d3f4ef8946e98d2e03f8194/users/3c01a452981647908bb4e1ace42d3bff/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:52.926590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2647a-9e99-4adc-911f-47be1f1aed9c None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:52.926928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2647a-9e99-4adc-911f-47be1f1aed9c None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:52.927159 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2647a-9e99-4adc-911f-47be1f1aed9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:52.945328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-adb2647a-9e99-4adc-911f-47be1f1aed9c None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:52.948730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-adb2647a-9e99-4adc-911f-47be1f1aed9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:52.956628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793f5d8-3b22-497b-8e74-7edb86ac219f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:53.005661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d369b4-e232-44f8-bf40-133154c61efe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:53.006266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d369b4-e232-44f8-bf40-133154c61efe service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.006544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d369b4-e232-44f8-bf40-133154c61efe service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.006653 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 796/1608] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:53.006806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d369b4-e232-44f8-bf40-133154c61efe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.007555 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2647a-9e99-4adc-911f-47be1f1aed9c None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:53.017388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2647a-9e99-4adc-911f-47be1f1aed9c None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:53.018139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d369b4-e232-44f8-bf40-133154c61efe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:53.019568 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d369b4-e232-44f8-bf40-133154c61efe service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:53.020978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fa43f-0a0c-4bcd-87f8-e91197ce0dbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:53.021581 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fa43f-0a0c-4bcd-87f8-e91197ce0dbc service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.021863 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fa43f-0a0c-4bcd-87f8-e91197ce0dbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.022135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fa43f-0a0c-4bcd-87f8-e91197ce0dbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.023570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:53.030219 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 813/1609] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:53.031063 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d369b4-e232-44f8-bf40-133154c61efe service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:53.031919 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fa43f-0a0c-4bcd-87f8-e91197ce0dbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:53.033382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fa43f-0a0c-4bcd-87f8-e91197ce0dbc service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:53.040110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d369b4-e232-44f8-bf40-133154c61efe service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:53.043914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 797/1610] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:53.044829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fa43f-0a0c-4bcd-87f8-e91197ce0dbc service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:53.045173 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f3fd0-0818-44ef-bd79-1451dafdd270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:53.045946 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f3fd0-0818-44ef-bd79-1451dafdd270 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.046229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f3fd0-0818-44ef-bd79-1451dafdd270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.046555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f3fd0-0818-44ef-bd79-1451dafdd270 admin admin] PATH_INFO: `/v3/projects/6b5b284e0b47422592057492601db382/users/88efd5eb09064aa2aa89e6d9580c6476/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.054510 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fa43f-0a0c-4bcd-87f8-e91197ce0dbc service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:53.059590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d94d97-7abb-4b77-8734-0301c2d3f690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:53.060248 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d94d97-7abb-4b77-8734-0301c2d3f690 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.060568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d94d97-7abb-4b77-8734-0301c2d3f690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.061128 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d94d97-7abb-4b77-8734-0301c2d3f690 admin admin] PATH_INFO: `/v3/projects/8302e73704074eae96d07513294b13b4/users/7421997e7abe42c9a25563d0e9dd266c/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.064922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f3fd0-0818-44ef-bd79-1451dafdd270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b5b284e0b47422592057492601db382, user_id=88efd5eb09064aa2aa89e6d9580c6476, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:53.067782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f3fd0-0818-44ef-bd79-1451dafdd270 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:53.078306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d94d97-7abb-4b77-8734-0301c2d3f690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8302e73704074eae96d07513294b13b4, user_id=7421997e7abe42c9a25563d0e9dd266c, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:53.081010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d94d97-7abb-4b77-8734-0301c2d3f690 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:53.168907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 798/1611] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:47:53 2022] PUT /identity/v3/projects/8302e73704074eae96d07513294b13b4/users/7421997e7abe42c9a25563d0e9dd266c/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:53.172545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d94d97-7abb-4b77-8734-0301c2d3f690 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:53.180786 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:53.181051 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 814/1612] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:47:53 2022] PUT /identity/v3/projects/6b5b284e0b47422592057492601db382/users/88efd5eb09064aa2aa89e6d9580c6476/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:53.183333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db6a22-6994-47bd-b424-b43425acae10 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.183719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db6a22-6994-47bd-b424-b43425acae10 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.184157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db6a22-6994-47bd-b424-b43425acae10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.216470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b0db6a22-6994-47bd-b424-b43425acae10 None None] MFA Rules not processed for user `f7bee141038c437eb57f3aa0e97558f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:53.217698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b0db6a22-6994-47bd-b424-b43425acae10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:53.237051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d94d97-7abb-4b77-8734-0301c2d3f690 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:53.294184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 815/1613] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:53.295357 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db6a22-6994-47bd-b424-b43425acae10 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:53.303326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb5e88-172d-4a29-a2c1-95f7a0617e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:53.304195 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb5e88-172d-4a29-a2c1-95f7a0617e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.309137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb5e88-172d-4a29-a2c1-95f7a0617e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.309137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb5e88-172d-4a29-a2c1-95f7a0617e96 admin admin] PATH_INFO: `/v3/projects/ae344977452b43acb4a5f6b1e4aafbbd/users/0e61fef0f7d2437eb97b796ee86cdc5c/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.321713 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb5e88-172d-4a29-a2c1-95f7a0617e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae344977452b43acb4a5f6b1e4aafbbd, user_id=0e61fef0f7d2437eb97b796ee86cdc5c, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:53.325297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb5e88-172d-4a29-a2c1-95f7a0617e96 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:53.362324 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db6a22-6994-47bd-b424-b43425acae10 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:53.366855 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd136a7-04a7-42d0-a2f3-088303805785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:53.367459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd136a7-04a7-42d0-a2f3-088303805785 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.372104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd136a7-04a7-42d0-a2f3-088303805785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.372104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd136a7-04a7-42d0-a2f3-088303805785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.372104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd136a7-04a7-42d0-a2f3-088303805785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:53.372104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd136a7-04a7-42d0-a2f3-088303805785 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:53.380443 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 816/1614] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:53.381236 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd136a7-04a7-42d0-a2f3-088303805785 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:53.382786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4cd136a7-04a7-42d0-a2f3-088303805785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:53.427284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 799/1615] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:47:53 2022] PUT /identity/v3/projects/ae344977452b43acb4a5f6b1e4aafbbd/users/0e61fef0f7d2437eb97b796ee86cdc5c/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:53.428370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585e073-395c-4780-8af0-b34ebdbd32c6 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.428704 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585e073-395c-4780-8af0-b34ebdbd32c6 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.429339 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585e073-395c-4780-8af0-b34ebdbd32c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.450234 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd136a7-04a7-42d0-a2f3-088303805785 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:53.463442 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:53.467469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-2585e073-395c-4780-8af0-b34ebdbd32c6 None None] MFA Rules not processed for user `7421997e7abe42c9a25563d0e9dd266c`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:53.469219 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2585e073-395c-4780-8af0-b34ebdbd32c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:53.508958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043d683-d1ba-440c-8327-1bb8ab148b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:53.509778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043d683-d1ba-440c-8327-1bb8ab148b86 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.510128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043d683-d1ba-440c-8327-1bb8ab148b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.510470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043d683-d1ba-440c-8327-1bb8ab148b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.552422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 800/1616] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:53.552422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c42995-406e-4484-9fc3-a87ffd85b506 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.552422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c42995-406e-4484-9fc3-a87ffd85b506 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.552422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c42995-406e-4484-9fc3-a87ffd85b506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.574566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-97c42995-406e-4484-9fc3-a87ffd85b506 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:53.575966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-97c42995-406e-4484-9fc3-a87ffd85b506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:53.602406 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:53.604857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043d683-d1ba-440c-8327-1bb8ab148b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:53.605884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043d683-d1ba-440c-8327-1bb8ab148b86 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:53.618886 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 817/1617] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:53.621180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043d683-d1ba-440c-8327-1bb8ab148b86 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:53.631846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043d683-d1ba-440c-8327-1bb8ab148b86 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:53.641830 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206abdce-e89d-4829-8ab6-98bd8522685f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:53.642455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206abdce-e89d-4829-8ab6-98bd8522685f admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.642734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206abdce-e89d-4829-8ab6-98bd8522685f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.643008 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206abdce-e89d-4829-8ab6-98bd8522685f admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.643883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206abdce-e89d-4829-8ab6-98bd8522685f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:53.645576 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206abdce-e89d-4829-8ab6-98bd8522685f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:53.653606 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 801/1618] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:53.654436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c42995-406e-4484-9fc3-a87ffd85b506 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:53.664733 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c42995-406e-4484-9fc3-a87ffd85b506 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:53.670168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71b280-7162-482c-9a29-a902865c5115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:53.670815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71b280-7162-482c-9a29-a902865c5115 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.671094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71b280-7162-482c-9a29-a902865c5115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.671366 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71b280-7162-482c-9a29-a902865c5115 admin admin] PATH_INFO: `/v3/projects/3f12cf6c0d3f4ef8946e98d2e03f8194/users/3c01a452981647908bb4e1ace42d3bff/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.673440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71b280-7162-482c-9a29-a902865c5115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f12cf6c0d3f4ef8946e98d2e03f8194, user_id=3c01a452981647908bb4e1ace42d3bff, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:53.676137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71b280-7162-482c-9a29-a902865c5115 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:53.687156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 802/1619] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:47:53 2022] PUT /identity/v3/projects/3f12cf6c0d3f4ef8946e98d2e03f8194/users/3c01a452981647908bb4e1ace42d3bff/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:53.688473 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec28d7-1d70-4e82-b600-28e2c1755d50 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.688764 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec28d7-1d70-4e82-b600-28e2c1755d50 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.689040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec28d7-1d70-4e82-b600-28e2c1755d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.718011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-8bec28d7-1d70-4e82-b600-28e2c1755d50 None None] MFA Rules not processed for user `0e61fef0f7d2437eb97b796ee86cdc5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:53.719230 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-8bec28d7-1d70-4e82-b600-28e2c1755d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:53.772999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 818/1620] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:47:53 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:53.772999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206abdce-e89d-4829-8ab6-98bd8522685f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:53.812891 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 803/1621] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:53.812891 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec28d7-1d70-4e82-b600-28e2c1755d50 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:53.880094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec28d7-1d70-4e82-b600-28e2c1755d50 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:53.934287 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206abdce-e89d-4829-8ab6-98bd8522685f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:53.941137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df078dd-2fce-4267-b48d-d9b83a7721c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:53.941896 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df078dd-2fce-4267-b48d-d9b83a7721c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.942279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df078dd-2fce-4267-b48d-d9b83a7721c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.942683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df078dd-2fce-4267-b48d-d9b83a7721c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.944827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f0a93-2f09-4cd5-af8e-55835dc3f7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:53.949094 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f0a93-2f09-4cd5-af8e-55835dc3f7fb admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.949094 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f0a93-2f09-4cd5-af8e-55835dc3f7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.949094 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f0a93-2f09-4cd5-af8e-55835dc3f7fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.949094 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f0a93-2f09-4cd5-af8e-55835dc3f7fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:53.949580 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f0a93-2f09-4cd5-af8e-55835dc3f7fb admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:53.953828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df078dd-2fce-4267-b48d-d9b83a7721c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:53.955398 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df078dd-2fce-4267-b48d-d9b83a7721c1 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:53.966247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 804/1622] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:53.967815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933cf82-0587-4c9a-bbdc-ff201c979dfe None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:53.968169 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933cf82-0587-4c9a-bbdc-ff201c979dfe None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:53.968550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933cf82-0587-4c9a-bbdc-ff201c979dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:53.977335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:54.001758 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-e933cf82-0587-4c9a-bbdc-ff201c979dfe None None] MFA Rules not processed for user `3c01a452981647908bb4e1ace42d3bff`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:54.003023 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e933cf82-0587-4c9a-bbdc-ff201c979dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:54.066753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 819/1623] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:47:53 2022] POST /identity/v3/projects => generated 385 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:54.067646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f0a93-2f09-4cd5-af8e-55835dc3f7fb admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:54.092763 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 805/1624] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:54.094205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ebebf-b7fb-4684-87f7-2fbb528fadb7 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:54.094551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ebebf-b7fb-4684-87f7-2fbb528fadb7 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:54.094833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ebebf-b7fb-4684-87f7-2fbb528fadb7 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:54.096561 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 806/1625] 10.222.0.70 () {54 vars in 962 bytes} [Wed Aug 24 16:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:47:54.098445 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ebebf-b7fb-4684-87f7-2fbb528fadb7 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:54.130000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:54.195302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f0a93-2f09-4cd5-af8e-55835dc3f7fb admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:54.196922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ebebf-b7fb-4684-87f7-2fbb528fadb7 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:54.202312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea61d4e-e721-4781-b808-d7466ebbe739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:54.203155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea61d4e-e721-4781-b808-d7466ebbe739 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:54.203658 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea61d4e-e721-4781-b808-d7466ebbe739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:54.203879 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb9ae9-8179-4bed-aaf5-f2b4c00eef87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:54.204150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea61d4e-e721-4781-b808-d7466ebbe739 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:54.205058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea61d4e-e721-4781-b808-d7466ebbe739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:54.205254 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb9ae9-8179-4bed-aaf5-f2b4c00eef87 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:54.206132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb9ae9-8179-4bed-aaf5-f2b4c00eef87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:54.206574 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb9ae9-8179-4bed-aaf5-f2b4c00eef87 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:54.207585 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb9ae9-8179-4bed-aaf5-f2b4c00eef87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:54.207916 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea61d4e-e721-4781-b808-d7466ebbe739 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:54.210678 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb9ae9-8179-4bed-aaf5-f2b4c00eef87 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:54.347235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 820/1627] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:47:54 2022] POST /identity/v3/users => generated 297 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:54.347455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 807/1627] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:47:54 2022] POST /identity/v3/users => generated 299 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:54.348275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb9ae9-8179-4bed-aaf5-f2b4c00eef87 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:54.349103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307b00e-c3a2-4f24-9cd1-70f77e00a3b5 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:54.349495 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307b00e-c3a2-4f24-9cd1-70f77e00a3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:54.349927 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307b00e-c3a2-4f24-9cd1-70f77e00a3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:54.417448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-9307b00e-c3a2-4f24-9cd1-70f77e00a3b5 None None] MFA Rules not processed for user `e333fca7790244f6a4ebceabb55faba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:54.418710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-9307b00e-c3a2-4f24-9cd1-70f77e00a3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:54.428677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb9ae9-8179-4bed-aaf5-f2b4c00eef87 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:54.432695 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998a8fa-1a6f-4f79-8c52-2b0698ac67bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:54.433371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998a8fa-1a6f-4f79-8c52-2b0698ac67bd service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:54.433659 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998a8fa-1a6f-4f79-8c52-2b0698ac67bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:54.433932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998a8fa-1a6f-4f79-8c52-2b0698ac67bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:54.535249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998a8fa-1a6f-4f79-8c52-2b0698ac67bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:54.537720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 808/1628] 10.222.0.70 () {58 vars in 1059 bytes} [Wed Aug 24 16:47:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:54.538042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998a8fa-1a6f-4f79-8c52-2b0698ac67bd service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:54.538667 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307b00e-c3a2-4f24-9cd1-70f77e00a3b5 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:54.555158 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 821/1629] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:54.556666 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998a8fa-1a6f-4f79-8c52-2b0698ac67bd service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:54.558174 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307b00e-c3a2-4f24-9cd1-70f77e00a3b5 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:54.563400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b7ce3-f45c-442f-8aa6-bb7afbd676ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:54.565516 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b7ce3-f45c-442f-8aa6-bb7afbd676ac admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:54.565947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b7ce3-f45c-442f-8aa6-bb7afbd676ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:54.566321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b7ce3-f45c-442f-8aa6-bb7afbd676ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:54.567096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b7ce3-f45c-442f-8aa6-bb7afbd676ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:54.569752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b7ce3-f45c-442f-8aa6-bb7afbd676ac admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:54.581019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998a8fa-1a6f-4f79-8c52-2b0698ac67bd service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:54.589215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 809/1630] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:54.590617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b7ce3-f45c-442f-8aa6-bb7afbd676ac admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:54.598079 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baaa3ec-774e-4aac-9a18-ea8d83e61763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:54.599023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaa3ec-774e-4aac-9a18-ea8d83e61763 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:54.600191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaa3ec-774e-4aac-9a18-ea8d83e61763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:54.600741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaa3ec-774e-4aac-9a18-ea8d83e61763 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:54.602024 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baaa3ec-774e-4aac-9a18-ea8d83e61763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:54.603699 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baaa3ec-774e-4aac-9a18-ea8d83e61763 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:54.607800 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b7ce3-f45c-442f-8aa6-bb7afbd676ac admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:54.613938 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b8f75-9bc9-4b74-9c1d-d3bc13e95e9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:54.614599 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b8f75-9bc9-4b74-9c1d-d3bc13e95e9a service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:54.614843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b8f75-9bc9-4b74-9c1d-d3bc13e95e9a service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:54.615282 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 822/1631] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:54.616164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baaa3ec-774e-4aac-9a18-ea8d83e61763 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:54.618630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b8f75-9bc9-4b74-9c1d-d3bc13e95e9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:54.627327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baaa3ec-774e-4aac-9a18-ea8d83e61763 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:54.632307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020232e-f864-43d0-ad8c-71618e201505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:54.633021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020232e-f864-43d0-ad8c-71618e201505 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:54.633302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020232e-f864-43d0-ad8c-71618e201505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:54.633841 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020232e-f864-43d0-ad8c-71618e201505 admin admin] PATH_INFO: `/v3/projects/91289101ca454ee0ba68947a2a96be54/users/bbd5b43e29af4a4585bcd4eceef56a12/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:54.653349 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020232e-f864-43d0-ad8c-71618e201505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91289101ca454ee0ba68947a2a96be54, user_id=bbd5b43e29af4a4585bcd4eceef56a12, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:54.656170 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020232e-f864-43d0-ad8c-71618e201505 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:54.667677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:54.702830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b8f75-9bc9-4b74-9c1d-d3bc13e95e9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:54.704488 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b8f75-9bc9-4b74-9c1d-d3bc13e95e9a service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:54.714039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 810/1632] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:54.714838 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b8f75-9bc9-4b74-9c1d-d3bc13e95e9a service nova] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:54.725811 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b8f75-9bc9-4b74-9c1d-d3bc13e95e9a service nova] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:54.731944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdfef8-def1-4a93-b4f1-f609bcb7c944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:54.732883 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdfef8-def1-4a93-b4f1-f609bcb7c944 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:54.733166 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdfef8-def1-4a93-b4f1-f609bcb7c944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:54.733441 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdfef8-def1-4a93-b4f1-f609bcb7c944 admin admin] PATH_INFO: `/v3/projects/11601975e0454e8e876595c5a1106ff4/users/ec158632ff71477e9e23b8ab0a9555fc/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:54.747022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 823/1633] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:47:54 2022] PUT /identity/v3/projects/91289101ca454ee0ba68947a2a96be54/users/bbd5b43e29af4a4585bcd4eceef56a12/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:54.751646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020232e-f864-43d0-ad8c-71618e201505 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:54.759063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdfef8-def1-4a93-b4f1-f609bcb7c944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11601975e0454e8e876595c5a1106ff4, user_id=ec158632ff71477e9e23b8ab0a9555fc, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:54.764102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdfef8-def1-4a93-b4f1-f609bcb7c944 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:54.831902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:54.839791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020232e-f864-43d0-ad8c-71618e201505 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:54.856416 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd32430-9452-4e93-929a-661cd0720f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:54.857523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd32430-9452-4e93-929a-661cd0720f66 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:54.857918 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd32430-9452-4e93-929a-661cd0720f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:54.858346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd32430-9452-4e93-929a-661cd0720f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:54.859745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd32430-9452-4e93-929a-661cd0720f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:54.862234 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd32430-9452-4e93-929a-661cd0720f66 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:54.882203 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 824/1634] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:54.889475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd32430-9452-4e93-929a-661cd0720f66 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:54.894728 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 811/1635] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:47:54 2022] PUT /identity/v3/projects/11601975e0454e8e876595c5a1106ff4/users/ec158632ff71477e9e23b8ab0a9555fc/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:54.898572 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdfef8-def1-4a93-b4f1-f609bcb7c944 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:54.994643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd32430-9452-4e93-929a-661cd0720f66 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:55.014136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdfef8-def1-4a93-b4f1-f609bcb7c944 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:55.020357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634d3d3-cdcd-4261-924a-c4b2271eee9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:55.020357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634d3d3-cdcd-4261-924a-c4b2271eee9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:55.020666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634d3d3-cdcd-4261-924a-c4b2271eee9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:55.021256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634d3d3-cdcd-4261-924a-c4b2271eee9a admin admin] PATH_INFO: `/v3/projects/91289101ca454ee0ba68947a2a96be54/users/bbd5b43e29af4a4585bcd4eceef56a12/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:55.023397 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634d3d3-cdcd-4261-924a-c4b2271eee9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=91289101ca454ee0ba68947a2a96be54, user_id=bbd5b43e29af4a4585bcd4eceef56a12, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:55.026808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634d3d3-cdcd-4261-924a-c4b2271eee9a admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:55.040281 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 812/1636] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:47:54 2022] PUT /identity/v3/projects/91289101ca454ee0ba68947a2a96be54/users/bbd5b43e29af4a4585bcd4eceef56a12/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:55.042482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634d3d3-cdcd-4261-924a-c4b2271eee9a admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:55.048630 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36ed45-9d57-4d78-aa64-1e4d1d7dfe7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:55.049479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36ed45-9d57-4d78-aa64-1e4d1d7dfe7c service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:55.049663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36ed45-9d57-4d78-aa64-1e4d1d7dfe7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:55.049993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36ed45-9d57-4d78-aa64-1e4d1d7dfe7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:55.123863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36ed45-9d57-4d78-aa64-1e4d1d7dfe7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:55.125523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36ed45-9d57-4d78-aa64-1e4d1d7dfe7c service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:55.139586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 825/1637] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:55.141175 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e572176-b141-4306-870e-e85918d749b1 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:55.141469 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e572176-b141-4306-870e-e85918d749b1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:55.141750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e572176-b141-4306-870e-e85918d749b1 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:55.144217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 826/1638] 10.222.0.70 () {54 vars in 912 bytes} [Wed Aug 24 16:47:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:47:55.145798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd862af3-8e1a-493b-b3d2-0a114475d8c6 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:55.146115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd862af3-8e1a-493b-b3d2-0a114475d8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:55.147680 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd862af3-8e1a-493b-b3d2-0a114475d8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:55.178430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634d3d3-cdcd-4261-924a-c4b2271eee9a admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:55.192324 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fd862af3-8e1a-493b-b3d2-0a114475d8c6 None None] MFA Rules not processed for user `bbd5b43e29af4a4585bcd4eceef56a12`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:55.194428 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fd862af3-8e1a-493b-b3d2-0a114475d8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:55.199172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b7523-b2a4-4226-9b4b-262d8eaf5269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:55.199172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b7523-b2a4-4226-9b4b-262d8eaf5269 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:55.199172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b7523-b2a4-4226-9b4b-262d8eaf5269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:55.199172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b7523-b2a4-4226-9b4b-262d8eaf5269 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:55.199595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b7523-b2a4-4226-9b4b-262d8eaf5269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:55.200464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b7523-b2a4-4226-9b4b-262d8eaf5269 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:55.222601 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 813/1639] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:55.225596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34440d-d103-45b0-a252-35965cc8c017 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:55.225596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34440d-d103-45b0-a252-35965cc8c017 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:55.235341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34440d-d103-45b0-a252-35965cc8c017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:55.278871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-db34440d-d103-45b0-a252-35965cc8c017 None None] MFA Rules not processed for user `e333fca7790244f6a4ebceabb55faba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:55.280103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-db34440d-d103-45b0-a252-35965cc8c017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:55.405260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 814/1640] 10.222.0.70 () {58 vars in 1009 bytes} [Wed Aug 24 16:47:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:55.407118 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34440d-d103-45b0-a252-35965cc8c017 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:55.422163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:55.434218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34440d-d103-45b0-a252-35965cc8c017 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:55.443914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b4c23-6176-42ad-b7a5-19dbe8ff80be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:55.445363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b4c23-6176-42ad-b7a5-19dbe8ff80be admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:55.445996 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b4c23-6176-42ad-b7a5-19dbe8ff80be admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:55.446624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b4c23-6176-42ad-b7a5-19dbe8ff80be admin admin] PATH_INFO: `/v3/projects/11601975e0454e8e876595c5a1106ff4/users/ec158632ff71477e9e23b8ab0a9555fc/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:55.454126 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b4c23-6176-42ad-b7a5-19dbe8ff80be admin admin] RBAC: Authorizing `identity:create_grant(project_id=11601975e0454e8e876595c5a1106ff4, user_id=ec158632ff71477e9e23b8ab0a9555fc, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:55.461759 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b4c23-6176-42ad-b7a5-19dbe8ff80be admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:55.481437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 827/1641] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:55.485005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd862af3-8e1a-493b-b3d2-0a114475d8c6 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:55.499370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 815/1642] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:47:55 2022] PUT /identity/v3/projects/11601975e0454e8e876595c5a1106ff4/users/ec158632ff71477e9e23b8ab0a9555fc/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:55.520204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af2dcc-fe3b-4469-805c-a95e1fe6998c None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:55.521352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af2dcc-fe3b-4469-805c-a95e1fe6998c None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:55.522086 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af2dcc-fe3b-4469-805c-a95e1fe6998c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:55.599133 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-45af2dcc-fe3b-4469-805c-a95e1fe6998c None None] MFA Rules not processed for user `ec158632ff71477e9e23b8ab0a9555fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:55.601377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-45af2dcc-fe3b-4469-805c-a95e1fe6998c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:55.679230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd862af3-8e1a-493b-b3d2-0a114475d8c6 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:55.833715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63ac6d-a037-4547-9914-fdcaaa50153b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:55.839249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63ac6d-a037-4547-9914-fdcaaa50153b service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:55.839249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63ac6d-a037-4547-9914-fdcaaa50153b service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:55.839249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63ac6d-a037-4547-9914-fdcaaa50153b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:55.839739 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 816/1643] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:55.876393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:55.969232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63ac6d-a037-4547-9914-fdcaaa50153b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:55.970954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63ac6d-a037-4547-9914-fdcaaa50153b service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:55.982665 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af2dcc-fe3b-4469-805c-a95e1fe6998c None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:55.991837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 828/1644] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:55.996212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af2dcc-fe3b-4469-805c-a95e1fe6998c None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:56.000689 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85e468-9700-4627-98b8-07159d73e8c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:56.001410 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85e468-9700-4627-98b8-07159d73e8c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:56.001826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85e468-9700-4627-98b8-07159d73e8c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:56.002456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85e468-9700-4627-98b8-07159d73e8c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:56.082115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85e468-9700-4627-98b8-07159d73e8c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:56.084113 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85e468-9700-4627-98b8-07159d73e8c2 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:56.099269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 817/1645] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:56.144811 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63ac6d-a037-4547-9914-fdcaaa50153b service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:56.157171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63ac6d-a037-4547-9914-fdcaaa50153b service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:56.164301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567f182-aa50-48bf-9427-7df536fa707f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:56.164966 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567f182-aa50-48bf-9427-7df536fa707f service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:56.165252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567f182-aa50-48bf-9427-7df536fa707f service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:56.165529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567f182-aa50-48bf-9427-7df536fa707f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:56.190475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b567f182-aa50-48bf-9427-7df536fa707f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:56.191847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b567f182-aa50-48bf-9427-7df536fa707f service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:56.216777 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 829/1646] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:56.265971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85e468-9700-4627-98b8-07159d73e8c2 service cinder] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:56.521572 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85e468-9700-4627-98b8-07159d73e8c2 service cinder] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:56.521572 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b6868-3716-42fb-bb9f-2853a70e908e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:56.521572 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b6868-3716-42fb-bb9f-2853a70e908e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:56.521572 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b6868-3716-42fb-bb9f-2853a70e908e admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:56.522075 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b6868-3716-42fb-bb9f-2853a70e908e admin admin] PATH_INFO: `/v3/users/7421997e7abe42c9a25563d0e9dd266c` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:56.522075 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b6868-3716-42fb-bb9f-2853a70e908e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7421997e7abe42c9a25563d0e9dd266c)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:56.522075 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: WARNING py.warnings [None req-b69b6868-3716-42fb-bb9f-2853a70e908e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:47:56.522075 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: warnings.warn(msg) Aug 24 16:47:56.522075 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: Aug 24 16:47:56.522075 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b6868-3716-42fb-bb9f-2853a70e908e admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:56.761655 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b69b6868-3716-42fb-bb9f-2853a70e908e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7421997e7abe42c9a25563d0e9dd266c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:56.762490 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b69b6868-3716-42fb-bb9f-2853a70e908e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7421997e7abe42c9a25563d0e9dd266c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:56.763208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b69b6868-3716-42fb-bb9f-2853a70e908e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7421997e7abe42c9a25563d0e9dd266c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:56.788749 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b69b6868-3716-42fb-bb9f-2853a70e908e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7421997e7abe42c9a25563d0e9dd266c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:56.882759 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b69b6868-3716-42fb-bb9f-2853a70e908e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7421997e7abe42c9a25563d0e9dd266c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:56.912796 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:56.952882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 818/1647] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:47:56 2022] DELETE /identity/v3/users/7421997e7abe42c9a25563d0e9dd266c => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:56.959551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939131ea-2a04-40d9-b0b5-df550a088d88 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:56.960428 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939131ea-2a04-40d9-b0b5-df550a088d88 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:56.961206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939131ea-2a04-40d9-b0b5-df550a088d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:57.003677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-939131ea-2a04-40d9-b0b5-df550a088d88 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:57.005761 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-939131ea-2a04-40d9-b0b5-df550a088d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:57.012697 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:57.114203 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 830/1648] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:57.143329 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b6868-3716-42fb-bb9f-2853a70e908e admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:57.144979 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b69b6868-3716-42fb-bb9f-2853a70e908e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:57.243664 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b6868-3716-42fb-bb9f-2853a70e908e admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:57.251234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b73f4-bfe2-4317-8f1d-c0392a686dcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:57.257041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b73f4-bfe2-4317-8f1d-c0392a686dcd service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:57.257345 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b73f4-bfe2-4317-8f1d-c0392a686dcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:57.257629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b73f4-bfe2-4317-8f1d-c0392a686dcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:57.286180 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b73f4-bfe2-4317-8f1d-c0392a686dcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:57.287930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b73f4-bfe2-4317-8f1d-c0392a686dcd service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:57.301574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 819/1649] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:57.528348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939131ea-2a04-40d9-b0b5-df550a088d88 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:57.530467 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-939131ea-2a04-40d9-b0b5-df550a088d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:57.639446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939131ea-2a04-40d9-b0b5-df550a088d88 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:57.644233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968f444-47ea-424b-bfa3-7285fd2e7198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:57.644951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968f444-47ea-424b-bfa3-7285fd2e7198 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:57.645233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968f444-47ea-424b-bfa3-7285fd2e7198 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:57.645510 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968f444-47ea-424b-bfa3-7285fd2e7198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:57.648480 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d968f444-47ea-424b-bfa3-7285fd2e7198 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:57.763634 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968f444-47ea-424b-bfa3-7285fd2e7198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:57.765705 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968f444-47ea-424b-bfa3-7285fd2e7198 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:57.789979 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 831/1650] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:57.928866 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35366a3-3c13-473e-a233-3bb5aa28d07a None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:57.929209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35366a3-3c13-473e-a233-3bb5aa28d07a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:57.929563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35366a3-3c13-473e-a233-3bb5aa28d07a None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:57.933970 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 820/1651] 10.222.0.70 () {54 vars in 962 bytes} [Wed Aug 24 16:47:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:47:57.943805 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f50b2f-052f-48f1-a6e6-5197ee9ba555 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:57.944128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f50b2f-052f-48f1-a6e6-5197ee9ba555 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:57.944469 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f50b2f-052f-48f1-a6e6-5197ee9ba555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:58.006627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-10f50b2f-052f-48f1-a6e6-5197ee9ba555 None None] MFA Rules not processed for user `e333fca7790244f6a4ebceabb55faba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:58.007816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-10f50b2f-052f-48f1-a6e6-5197ee9ba555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:58.101131 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:58.127547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 832/1652] 10.222.0.70 () {58 vars in 1059 bytes} [Wed Aug 24 16:47:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:58.133680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b5cdb-627e-4db4-b7e0-76971f1206b1 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:58.134457 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b5cdb-627e-4db4-b7e0-76971f1206b1 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:58.135112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b5cdb-627e-4db4-b7e0-76971f1206b1 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:58.137778 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 821/1653] 10.222.0.70 () {54 vars in 912 bytes} [Wed Aug 24 16:47:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:47:58.143651 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2c124-6c48-43f5-9f99-dc9ec4a08a7a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:58.144340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2c124-6c48-43f5-9f99-dc9ec4a08a7a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:58.144870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2c124-6c48-43f5-9f99-dc9ec4a08a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:58.194435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-48e2c124-6c48-43f5-9f99-dc9ec4a08a7a None None] MFA Rules not processed for user `e333fca7790244f6a4ebceabb55faba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:58.196187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-48e2c124-6c48-43f5-9f99-dc9ec4a08a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:58.222295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 833/1654] 10.222.0.70 () {58 vars in 1009 bytes} [Wed Aug 24 16:47:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:58.233342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b5cdb-627e-4db4-b7e0-76971f1206b1 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:58.237067 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-515b5cdb-627e-4db4-b7e0-76971f1206b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:58.261226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b5cdb-627e-4db4-b7e0-76971f1206b1 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:58.277290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58986c6f-6b9c-4471-97d3-a52ccb35f636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:58.278132 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58986c6f-6b9c-4471-97d3-a52ccb35f636 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:58.278563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58986c6f-6b9c-4471-97d3-a52ccb35f636 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:58.278955 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58986c6f-6b9c-4471-97d3-a52ccb35f636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:58.300697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58986c6f-6b9c-4471-97d3-a52ccb35f636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:58.302395 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58986c6f-6b9c-4471-97d3-a52ccb35f636 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:58.308119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:58.319331 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 822/1655] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:58.689067 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2c124-6c48-43f5-9f99-dc9ec4a08a7a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:58.691159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-48e2c124-6c48-43f5-9f99-dc9ec4a08a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:58.742402 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2c124-6c48-43f5-9f99-dc9ec4a08a7a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:58.783566 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:58.784247 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:58.787375 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:58.787693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] PATH_INFO: `/v3/projects/8302e73704074eae96d07513294b13b4` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:58.794841 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8302e73704074eae96d07513294b13b4)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:58.796005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: WARNING py.warnings [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:47:58.796005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: warnings.warn(msg) Aug 24 16:47:58.796005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: Aug 24 16:47:58.797534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:59.103860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8302e73704074eae96d07513294b13b4', 'request_id': 'req-eefa1614-1fab-4767-8de5-47c255725253'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:59.159821 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8302e73704074eae96d07513294b13b4', 'request_id': 'req-eefa1614-1fab-4767-8de5-47c255725253'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:59.175558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8302e73704074eae96d07513294b13b4', 'request_id': 'req-eefa1614-1fab-4767-8de5-47c255725253'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:59.304655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] The token cache is being invalidate because project 8302e73704074eae96d07513294b13b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:47:59.305870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:59.307093 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:59.313888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 834/1656] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:47:58 2022] DELETE /identity/v3/projects/8302e73704074eae96d07513294b13b4 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:59.315174 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58986c6f-6b9c-4471-97d3-a52ccb35f636 service cinder] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:59.326470 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58986c6f-6b9c-4471-97d3-a52ccb35f636 service cinder] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:59.475888 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759d648-8b27-4b16-8bec-434ff8f80806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:59.477906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759d648-8b27-4b16-8bec-434ff8f80806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:59.482083 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759d648-8b27-4b16-8bec-434ff8f80806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:59.482679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759d648-8b27-4b16-8bec-434ff8f80806 admin admin] PATH_INFO: `/v3/users/ec158632ff71477e9e23b8ab0a9555fc` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:59.485409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759d648-8b27-4b16-8bec-434ff8f80806 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec158632ff71477e9e23b8ab0a9555fc)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:59.487523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759d648-8b27-4b16-8bec-434ff8f80806 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:59.763685 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9759d648-8b27-4b16-8bec-434ff8f80806 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec158632ff71477e9e23b8ab0a9555fc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:59.764564 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9759d648-8b27-4b16-8bec-434ff8f80806 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec158632ff71477e9e23b8ab0a9555fc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:59.770657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9759d648-8b27-4b16-8bec-434ff8f80806 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec158632ff71477e9e23b8ab0a9555fc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:59.789687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9759d648-8b27-4b16-8bec-434ff8f80806 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec158632ff71477e9e23b8ab0a9555fc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:59.905453 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9759d648-8b27-4b16-8bec-434ff8f80806 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec158632ff71477e9e23b8ab0a9555fc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:59.948809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 823/1657] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:47:59 2022] DELETE /identity/v3/users/ec158632ff71477e9e23b8ab0a9555fc => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:01.586239 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:01.587702 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:01.614794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa1614-1fab-4767-8de5-47c255725253 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:01.802155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a0f18-0238-427a-997a-c38ae74189da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:01.802155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a0f18-0238-427a-997a-c38ae74189da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:01.802155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a0f18-0238-427a-997a-c38ae74189da admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:01.802155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a0f18-0238-427a-997a-c38ae74189da admin admin] PATH_INFO: `/v3/users/0e61fef0f7d2437eb97b796ee86cdc5c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:01.805775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a0f18-0238-427a-997a-c38ae74189da admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e61fef0f7d2437eb97b796ee86cdc5c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:01.811599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a0f18-0238-427a-997a-c38ae74189da admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:02.006501 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759d648-8b27-4b16-8bec-434ff8f80806 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:02.007966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-9759d648-8b27-4b16-8bec-434ff8f80806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:02.035275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759d648-8b27-4b16-8bec-434ff8f80806 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:02.051171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcdfb61-85a9-4615-b902-65b9066e095c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:02.052074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcdfb61-85a9-4615-b902-65b9066e095c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:02.053841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcdfb61-85a9-4615-b902-65b9066e095c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:02.054289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcdfb61-85a9-4615-b902-65b9066e095c admin admin] PATH_INFO: `/v3/projects/11601975e0454e8e876595c5a1106ff4` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:02.060239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcdfb61-85a9-4615-b902-65b9066e095c admin admin] RBAC: Authorizing `identity:delete_project(project_id=11601975e0454e8e876595c5a1106ff4)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:02.064370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcdfb61-85a9-4615-b902-65b9066e095c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:02.201042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8f3a0f18-0238-427a-997a-c38ae74189da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e61fef0f7d2437eb97b796ee86cdc5c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:02.202534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8f3a0f18-0238-427a-997a-c38ae74189da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e61fef0f7d2437eb97b796ee86cdc5c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:02.204279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8f3a0f18-0238-427a-997a-c38ae74189da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e61fef0f7d2437eb97b796ee86cdc5c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:02.319112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8f3a0f18-0238-427a-997a-c38ae74189da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e61fef0f7d2437eb97b796ee86cdc5c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:02.320034 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7dcdfb61-85a9-4615-b902-65b9066e095c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11601975e0454e8e876595c5a1106ff4', 'request_id': 'req-7dcdfb61-85a9-4615-b902-65b9066e095c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:02.333167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8f3a0f18-0238-427a-997a-c38ae74189da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e61fef0f7d2437eb97b796ee86cdc5c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:02.335832 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7dcdfb61-85a9-4615-b902-65b9066e095c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11601975e0454e8e876595c5a1106ff4', 'request_id': 'req-7dcdfb61-85a9-4615-b902-65b9066e095c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:02.346150 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:02.361729 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7dcdfb61-85a9-4615-b902-65b9066e095c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11601975e0454e8e876595c5a1106ff4', 'request_id': 'req-7dcdfb61-85a9-4615-b902-65b9066e095c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:02.370396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 835/1658] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:48:01 2022] DELETE /identity/v3/users/0e61fef0f7d2437eb97b796ee86cdc5c => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:02.372112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ceda78-2f58-4e12-bb71-17017e8f1243 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:02.372426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ceda78-2f58-4e12-bb71-17017e8f1243 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:02.387719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ceda78-2f58-4e12-bb71-17017e8f1243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:02.415843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b7ceda78-2f58-4e12-bb71-17017e8f1243 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:02.415843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b7ceda78-2f58-4e12-bb71-17017e8f1243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:02.478078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:02.487958 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7dcdfb61-85a9-4615-b902-65b9066e095c admin admin] The token cache is being invalidate because project 11601975e0454e8e876595c5a1106ff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:48:02.487958 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7dcdfb61-85a9-4615-b902-65b9066e095c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:02.490068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7dcdfb61-85a9-4615-b902-65b9066e095c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:02.491764 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 824/1659] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:48:02 2022] DELETE /identity/v3/projects/11601975e0454e8e876595c5a1106ff4 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:02.496874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf31959-0f8f-48ef-aaf2-f33619433cf4 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:02.497153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf31959-0f8f-48ef-aaf2-f33619433cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:02.497433 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf31959-0f8f-48ef-aaf2-f33619433cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:02.524243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 836/1660] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:02.549537 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ceda78-2f58-4e12-bb71-17017e8f1243 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:02.554018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-4cf31959-0f8f-48ef-aaf2-f33619433cf4 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:02.554895 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b7ceda78-2f58-4e12-bb71-17017e8f1243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:02.555228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-4cf31959-0f8f-48ef-aaf2-f33619433cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:02.592720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 825/1661] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:48:02.600797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74c3c7-925d-49d2-9678-62c01a6ab230 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:02.601315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74c3c7-925d-49d2-9678-62c01a6ab230 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:02.601730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74c3c7-925d-49d2-9678-62c01a6ab230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:02.645759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ceda78-2f58-4e12-bb71-17017e8f1243 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:02.651549 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f52bcf-1ea1-46c2-a600-ea0ad4018e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:02.652601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f52bcf-1ea1-46c2-a600-ea0ad4018e07 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:02.653092 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f52bcf-1ea1-46c2-a600-ea0ad4018e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:02.653586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f52bcf-1ea1-46c2-a600-ea0ad4018e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:02.664306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f52bcf-1ea1-46c2-a600-ea0ad4018e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:02.666177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f52bcf-1ea1-46c2-a600-ea0ad4018e07 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:02.672795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-2c74c3c7-925d-49d2-9678-62c01a6ab230 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:02.675371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2c74c3c7-925d-49d2-9678-62c01a6ab230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:02.677736 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 837/1662] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:02.692976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 826/1663] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:48:02.698074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f52bcf-1ea1-46c2-a600-ea0ad4018e07 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:02.710813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f52bcf-1ea1-46c2-a600-ea0ad4018e07 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:02.716448 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b80706-6ece-460f-8940-b7a009db7fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:02.717261 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b80706-6ece-460f-8940-b7a009db7fea admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:02.717687 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b80706-6ece-460f-8940-b7a009db7fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:02.720012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b80706-6ece-460f-8940-b7a009db7fea admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:02.720012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b80706-6ece-460f-8940-b7a009db7fea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:48:02.720242 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b80706-6ece-460f-8940-b7a009db7fea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:02.722181 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b80706-6ece-460f-8940-b7a009db7fea admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:02.736889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 838/1664] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:48:02.741012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74c3c7-925d-49d2-9678-62c01a6ab230 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:02.751089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74c3c7-925d-49d2-9678-62c01a6ab230 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:02.759274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1eee0-8104-44bc-b0e5-b9a739caedb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:02.762928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1eee0-8104-44bc-b0e5-b9a739caedb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:02.762928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1eee0-8104-44bc-b0e5-b9a739caedb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:02.762928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1eee0-8104-44bc-b0e5-b9a739caedb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:02.764007 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1eee0-8104-44bc-b0e5-b9a739caedb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:02.771407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1eee0-8104-44bc-b0e5-b9a739caedb9 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:02.895862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 827/1665] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:48:02 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:02.899887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b80706-6ece-460f-8940-b7a009db7fea admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:02.911921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b80706-6ece-460f-8940-b7a009db7fea admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:03.037191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab07c73-bc01-486b-87d9-e6022e4353c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:03.037727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab07c73-bc01-486b-87d9-e6022e4353c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:03.038013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab07c73-bc01-486b-87d9-e6022e4353c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:03.038540 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab07c73-bc01-486b-87d9-e6022e4353c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:03.039192 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab07c73-bc01-486b-87d9-e6022e4353c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:03.045447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab07c73-bc01-486b-87d9-e6022e4353c9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:03.163301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 839/1666] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:48:02 2022] POST /identity/v3/users => generated 294 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:48:03.171236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1eee0-8104-44bc-b0e5-b9a739caedb9 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:03.194319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1eee0-8104-44bc-b0e5-b9a739caedb9 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:03.208136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b548b4-990a-4386-8eaa-a7dbb1a6c047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:03.208929 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b548b4-990a-4386-8eaa-a7dbb1a6c047 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:03.209336 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b548b4-990a-4386-8eaa-a7dbb1a6c047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:03.209846 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b548b4-990a-4386-8eaa-a7dbb1a6c047 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:03.210716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b548b4-990a-4386-8eaa-a7dbb1a6c047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:03.212494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b548b4-990a-4386-8eaa-a7dbb1a6c047 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:03.258967 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 828/1667] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:03.264081 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab07c73-bc01-486b-87d9-e6022e4353c9 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:03.283915 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab07c73-bc01-486b-87d9-e6022e4353c9 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:03.290710 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a859109-ccc0-462b-92fd-af2815f12518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:03.291555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a859109-ccc0-462b-92fd-af2815f12518 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:03.291999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a859109-ccc0-462b-92fd-af2815f12518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:03.292628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a859109-ccc0-462b-92fd-af2815f12518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:03.293423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a859109-ccc0-462b-92fd-af2815f12518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:03.295371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a859109-ccc0-462b-92fd-af2815f12518 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:03.323367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 840/1668] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:48:03.324981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b548b4-990a-4386-8eaa-a7dbb1a6c047 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:03.348410 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b548b4-990a-4386-8eaa-a7dbb1a6c047 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:03.377090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b02f9d-6398-4bb1-8daf-43c494de50d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:03.378022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b02f9d-6398-4bb1-8daf-43c494de50d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:03.378455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b02f9d-6398-4bb1-8daf-43c494de50d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:03.381218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b02f9d-6398-4bb1-8daf-43c494de50d3 admin admin] PATH_INFO: `/v3/projects/3af266d023b7412c8b878b15b5fff053/users/c1f4a9bf7d14415e9c883fd200d47cf6/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:03.404295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b02f9d-6398-4bb1-8daf-43c494de50d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3af266d023b7412c8b878b15b5fff053, user_id=c1f4a9bf7d14415e9c883fd200d47cf6, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:03.408385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b02f9d-6398-4bb1-8daf-43c494de50d3 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:03.545413 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 829/1669] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:03 2022] PUT /identity/v3/projects/3af266d023b7412c8b878b15b5fff053/users/c1f4a9bf7d14415e9c883fd200d47cf6/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:03.558802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf471b-cd6b-4c0f-a5a3-9fb3b814ae3b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:03.559084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf471b-cd6b-4c0f-a5a3-9fb3b814ae3b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:03.559360 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf471b-cd6b-4c0f-a5a3-9fb3b814ae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:03.632732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-dedf471b-cd6b-4c0f-a5a3-9fb3b814ae3b None None] MFA Rules not processed for user `c1f4a9bf7d14415e9c883fd200d47cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:03.634151 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dedf471b-cd6b-4c0f-a5a3-9fb3b814ae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:03.804936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 841/1670] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:03.806512 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b02f9d-6398-4bb1-8daf-43c494de50d3 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:03.935231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b02f9d-6398-4bb1-8daf-43c494de50d3 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:03.935231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9c1b9-5465-4e01-9afc-3428867ad6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:03.935231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9c1b9-5465-4e01-9afc-3428867ad6ae admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:03.935231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9c1b9-5465-4e01-9afc-3428867ad6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:03.935552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9c1b9-5465-4e01-9afc-3428867ad6ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:03.936531 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9c1b9-5465-4e01-9afc-3428867ad6ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:03.939175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9c1b9-5465-4e01-9afc-3428867ad6ae admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:04.074558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:04.082892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 830/1671] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:48:03 2022] POST /identity/v3/projects => generated 387 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:04.087427 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf471b-cd6b-4c0f-a5a3-9fb3b814ae3b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:04.107512 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf471b-cd6b-4c0f-a5a3-9fb3b814ae3b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:04.228687 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec763e6-46ed-48c5-b2f9-356e6a70646e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:04.229340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec763e6-46ed-48c5-b2f9-356e6a70646e admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:04.229617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec763e6-46ed-48c5-b2f9-356e6a70646e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:04.229899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec763e6-46ed-48c5-b2f9-356e6a70646e admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:04.230535 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec763e6-46ed-48c5-b2f9-356e6a70646e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:04.232608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec763e6-46ed-48c5-b2f9-356e6a70646e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:04.362189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 842/1672] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:48:04 2022] POST /identity/v3/users => generated 300 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:04.366369 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9c1b9-5465-4e01-9afc-3428867ad6ae admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:04.392879 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9c1b9-5465-4e01-9afc-3428867ad6ae admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:04.408925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab1d7a-74ce-4217-9b66-31fb1b6af403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:04.409767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab1d7a-74ce-4217-9b66-31fb1b6af403 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:04.410241 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab1d7a-74ce-4217-9b66-31fb1b6af403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:04.410690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab1d7a-74ce-4217-9b66-31fb1b6af403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:04.411634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab1d7a-74ce-4217-9b66-31fb1b6af403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:04.413626 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab1d7a-74ce-4217-9b66-31fb1b6af403 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:04.437203 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 831/1673] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:04.439051 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec763e6-46ed-48c5-b2f9-356e6a70646e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:04.463409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec763e6-46ed-48c5-b2f9-356e6a70646e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:04.478606 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bfd6f-4bf0-40b3-bee7-bf003f942fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:04.479605 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bfd6f-4bf0-40b3-bee7-bf003f942fe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:04.480097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bfd6f-4bf0-40b3-bee7-bf003f942fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:04.480557 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bfd6f-4bf0-40b3-bee7-bf003f942fe3 admin admin] PATH_INFO: `/v3/projects/fd95ac269f3442308000bc994cacfe4e/users/ea19f29fa4ae42b297a29e90484bc365/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:04.518514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bfd6f-4bf0-40b3-bee7-bf003f942fe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd95ac269f3442308000bc994cacfe4e, user_id=ea19f29fa4ae42b297a29e90484bc365, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:04.521296 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bfd6f-4bf0-40b3-bee7-bf003f942fe3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:04.621159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 843/1674] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:04 2022] PUT /identity/v3/projects/fd95ac269f3442308000bc994cacfe4e/users/ea19f29fa4ae42b297a29e90484bc365/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:04.625062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab1d7a-74ce-4217-9b66-31fb1b6af403 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:04.641988 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab1d7a-74ce-4217-9b66-31fb1b6af403 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:04.796193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549347f-0afd-417a-9d4b-19bf01d83a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:04.796986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549347f-0afd-417a-9d4b-19bf01d83a17 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:04.797374 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549347f-0afd-417a-9d4b-19bf01d83a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:04.797755 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549347f-0afd-417a-9d4b-19bf01d83a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:04.798520 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549347f-0afd-417a-9d4b-19bf01d83a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:04.800857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549347f-0afd-417a-9d4b-19bf01d83a17 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:04.812776 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:04.835741 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 832/1675] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:04.836302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bfd6f-4bf0-40b3-bee7-bf003f942fe3 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:04.855331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bfd6f-4bf0-40b3-bee7-bf003f942fe3 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:04.871793 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301de6cc-4557-4bcf-aa3e-c1c7730e76e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:04.873213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301de6cc-4557-4bcf-aa3e-c1c7730e76e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:04.873495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301de6cc-4557-4bcf-aa3e-c1c7730e76e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:04.875570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301de6cc-4557-4bcf-aa3e-c1c7730e76e2 admin admin] PATH_INFO: `/v3/projects/fd95ac269f3442308000bc994cacfe4e/users/ea19f29fa4ae42b297a29e90484bc365/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:04.886084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301de6cc-4557-4bcf-aa3e-c1c7730e76e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd95ac269f3442308000bc994cacfe4e, user_id=ea19f29fa4ae42b297a29e90484bc365, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:04.893371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301de6cc-4557-4bcf-aa3e-c1c7730e76e2 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:04.929360 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 844/1676] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:04 2022] PUT /identity/v3/projects/fd95ac269f3442308000bc994cacfe4e/users/ea19f29fa4ae42b297a29e90484bc365/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:04.934094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec3354-9bb9-48b8-b004-1cacdf5a6db3 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:04.934094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec3354-9bb9-48b8-b004-1cacdf5a6db3 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:04.934094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec3354-9bb9-48b8-b004-1cacdf5a6db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:05.017359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-b4ec3354-9bb9-48b8-b004-1cacdf5a6db3 None None] MFA Rules not processed for user `ea19f29fa4ae42b297a29e90484bc365`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:05.021976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b4ec3354-9bb9-48b8-b004-1cacdf5a6db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:05.262023 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 833/1677] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:48:05.273096 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301de6cc-4557-4bcf-aa3e-c1c7730e76e2 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:05.274990 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-301de6cc-4557-4bcf-aa3e-c1c7730e76e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:05.321523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7829c-2487-4c8b-8c5a-1f45275a489d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:05.322014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7829c-2487-4c8b-8c5a-1f45275a489d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:05.323054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7829c-2487-4c8b-8c5a-1f45275a489d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:05.393019 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-73b7829c-2487-4c8b-8c5a-1f45275a489d None None] MFA Rules not processed for user `e2257adc34324452a325bb7786d02515`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:05.397106 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-73b7829c-2487-4c8b-8c5a-1f45275a489d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:05.429082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301de6cc-4557-4bcf-aa3e-c1c7730e76e2 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:05.439266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81dccab-131f-4135-9cf0-5861672423fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:05.441266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81dccab-131f-4135-9cf0-5861672423fc service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:05.441725 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81dccab-131f-4135-9cf0-5861672423fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:05.442156 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81dccab-131f-4135-9cf0-5861672423fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:05.456954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:05.459228 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81dccab-131f-4135-9cf0-5861672423fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:05.462073 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81dccab-131f-4135-9cf0-5861672423fc service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:05.491533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 845/1678] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:05 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:05.578182 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 834/1679] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:05.589678 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e227d59-da2a-49d4-8119-13a774696264 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:05.589970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e227d59-da2a-49d4-8119-13a774696264 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:05.590470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e227d59-da2a-49d4-8119-13a774696264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:05.633978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4e227d59-da2a-49d4-8119-13a774696264 None None] MFA Rules not processed for user `e2257adc34324452a325bb7786d02515`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:05.635930 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4e227d59-da2a-49d4-8119-13a774696264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:05.659571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 846/1680] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:05.668189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7829c-2487-4c8b-8c5a-1f45275a489d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:05.672092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-73b7829c-2487-4c8b-8c5a-1f45275a489d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:05.738291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e227d59-da2a-49d4-8119-13a774696264 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:05.746066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4e227d59-da2a-49d4-8119-13a774696264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:05.877111 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e227d59-da2a-49d4-8119-13a774696264 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:05.891065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338283b5-558d-41c8-ae20-e1cd93858e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:05.891065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338283b5-558d-41c8-ae20-e1cd93858e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:05.891065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338283b5-558d-41c8-ae20-e1cd93858e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:05.891065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338283b5-558d-41c8-ae20-e1cd93858e0c admin admin] PATH_INFO: `/v3/projects/ae344977452b43acb4a5f6b1e4aafbbd` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:05.891324 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338283b5-558d-41c8-ae20-e1cd93858e0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae344977452b43acb4a5f6b1e4aafbbd)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:05.899509 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7829c-2487-4c8b-8c5a-1f45275a489d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:05.901753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338283b5-558d-41c8-ae20-e1cd93858e0c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:05.933643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ce7b8-b427-43de-b217-1e2f464fe7ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:05.934786 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ce7b8-b427-43de-b217-1e2f464fe7ca service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:05.935072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ce7b8-b427-43de-b217-1e2f464fe7ca service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:05.935358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ce7b8-b427-43de-b217-1e2f464fe7ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:05.959616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ce7b8-b427-43de-b217-1e2f464fe7ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:05.963603 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ce7b8-b427-43de-b217-1e2f464fe7ca service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:05.974295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:05.982396 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 835/1681] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:06.068840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-338283b5-558d-41c8-ae20-e1cd93858e0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae344977452b43acb4a5f6b1e4aafbbd', 'request_id': 'req-338283b5-558d-41c8-ae20-e1cd93858e0c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:06.083457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-338283b5-558d-41c8-ae20-e1cd93858e0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae344977452b43acb4a5f6b1e4aafbbd', 'request_id': 'req-338283b5-558d-41c8-ae20-e1cd93858e0c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:06.101512 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-338283b5-558d-41c8-ae20-e1cd93858e0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae344977452b43acb4a5f6b1e4aafbbd', 'request_id': 'req-338283b5-558d-41c8-ae20-e1cd93858e0c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:06.231197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-338283b5-558d-41c8-ae20-e1cd93858e0c admin admin] The token cache is being invalidate because project ae344977452b43acb4a5f6b1e4aafbbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:48:06.238237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-338283b5-558d-41c8-ae20-e1cd93858e0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:06.238237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-338283b5-558d-41c8-ae20-e1cd93858e0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:06.238237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 847/1682] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:48:05 2022] DELETE /identity/v3/projects/ae344977452b43acb4a5f6b1e4aafbbd => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:06.239603 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6de441-5ed9-4399-a0bb-313424547032 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:06.239894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6de441-5ed9-4399-a0bb-313424547032 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:06.240167 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6de441-5ed9-4399-a0bb-313424547032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:06.282118 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-9d6de441-5ed9-4399-a0bb-313424547032 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:06.284580 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-9d6de441-5ed9-4399-a0bb-313424547032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:06.449705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 836/1683] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:06.455575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063aef9-f478-4d9e-8944-4d0fc6d3d63e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:06.462455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063aef9-f478-4d9e-8944-4d0fc6d3d63e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:06.462737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063aef9-f478-4d9e-8944-4d0fc6d3d63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:06.507180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8063aef9-f478-4d9e-8944-4d0fc6d3d63e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:06.521755 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8063aef9-f478-4d9e-8944-4d0fc6d3d63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:06.548307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 848/1684] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:06.554036 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6de441-5ed9-4399-a0bb-313424547032 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:06.575315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6de441-5ed9-4399-a0bb-313424547032 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:06.581751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15eae9f-b786-4203-aa00-1e2c22f0fe06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:06.582634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15eae9f-b786-4203-aa00-1e2c22f0fe06 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:06.583085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15eae9f-b786-4203-aa00-1e2c22f0fe06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:06.583512 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15eae9f-b786-4203-aa00-1e2c22f0fe06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:06.584344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15eae9f-b786-4203-aa00-1e2c22f0fe06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:48:06.584846 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15eae9f-b786-4203-aa00-1e2c22f0fe06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:06.586660 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15eae9f-b786-4203-aa00-1e2c22f0fe06 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:06.605588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 837/1685] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:48:06.608136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063aef9-f478-4d9e-8944-4d0fc6d3d63e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:06.623091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063aef9-f478-4d9e-8944-4d0fc6d3d63e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:06.630598 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b551d47-1963-4bcf-99ea-a7aaaa9aedbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:06.641200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b551d47-1963-4bcf-99ea-a7aaaa9aedbd admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:06.641890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b551d47-1963-4bcf-99ea-a7aaaa9aedbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:06.642367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b551d47-1963-4bcf-99ea-a7aaaa9aedbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:06.643266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b551d47-1963-4bcf-99ea-a7aaaa9aedbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:06.648572 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b551d47-1963-4bcf-99ea-a7aaaa9aedbd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:06.768464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 849/1686] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:48:06 2022] POST /identity/v3/projects => generated 405 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:06.770179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15eae9f-b786-4203-aa00-1e2c22f0fe06 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:06.784241 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15eae9f-b786-4203-aa00-1e2c22f0fe06 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:06.960836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb360e2-778e-480f-9d89-39a7a33b3b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:06.962588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb360e2-778e-480f-9d89-39a7a33b3b65 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:06.963115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb360e2-778e-480f-9d89-39a7a33b3b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:06.963595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb360e2-778e-480f-9d89-39a7a33b3b65 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:06.968771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb360e2-778e-480f-9d89-39a7a33b3b65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:06.971560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb360e2-778e-480f-9d89-39a7a33b3b65 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:07.186028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 838/1687] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:48:06 2022] POST /identity/v3/users => generated 303 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:07.189962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b551d47-1963-4bcf-99ea-a7aaaa9aedbd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:07.207783 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b551d47-1963-4bcf-99ea-a7aaaa9aedbd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:07.232102 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a19bc-c466-492c-9560-e7d41422431b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:07.234422 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a19bc-c466-492c-9560-e7d41422431b admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:07.234736 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a19bc-c466-492c-9560-e7d41422431b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:07.235019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a19bc-c466-492c-9560-e7d41422431b admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:07.235776 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a19bc-c466-492c-9560-e7d41422431b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:07.248796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a19bc-c466-492c-9560-e7d41422431b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:07.271500 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb360e2-778e-480f-9d89-39a7a33b3b65 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:07.288712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb360e2-778e-480f-9d89-39a7a33b3b65 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:07.289138 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 850/1688] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:07.291538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a54282-1b3c-4eb0-8b88-0594ead600e2 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:07.293378 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a54282-1b3c-4eb0-8b88-0594ead600e2 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:07.295104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a54282-1b3c-4eb0-8b88-0594ead600e2 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:07.299998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b6595-9613-43cd-9f36-e1c492a91d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:07.301102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b6595-9613-43cd-9f36-e1c492a91d99 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:07.301386 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b6595-9613-43cd-9f36-e1c492a91d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:07.301657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b6595-9613-43cd-9f36-e1c492a91d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:07.302284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 851/1689] 10.222.0.70 () {54 vars in 912 bytes} [Wed Aug 24 16:48:07 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:48:07.303006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b6595-9613-43cd-9f36-e1c492a91d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:07.304873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b6595-9613-43cd-9f36-e1c492a91d99 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:07.309693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b5cec-fd2b-4507-ab18-bfaf7af410ee None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:07.310270 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b5cec-fd2b-4507-ab18-bfaf7af410ee None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:07.311023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b5cec-fd2b-4507-ab18-bfaf7af410ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:07.380931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 839/1690] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:07.401372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b6595-9613-43cd-9f36-e1c492a91d99 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:07.416788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b6595-9613-43cd-9f36-e1c492a91d99 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:07.426952 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4417524-fcdd-44ff-9ba4-4d14c1b42c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:07.428756 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4417524-fcdd-44ff-9ba4-4d14c1b42c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:07.429232 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4417524-fcdd-44ff-9ba4-4d14c1b42c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:07.430016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4417524-fcdd-44ff-9ba4-4d14c1b42c20 admin admin] PATH_INFO: `/v3/projects/a8af09e38d54478397dd2f346c5d968c/users/6e8412ec926042b4ab513a82f0be8644/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:07.439450 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-539b5cec-fd2b-4507-ab18-bfaf7af410ee None None] MFA Rules not processed for user `942b54c39ae5403c8ae3bcca1c31bf1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:07.444755 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-539b5cec-fd2b-4507-ab18-bfaf7af410ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:07.455822 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4417524-fcdd-44ff-9ba4-4d14c1b42c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8af09e38d54478397dd2f346c5d968c, user_id=6e8412ec926042b4ab513a82f0be8644, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:07.458945 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4417524-fcdd-44ff-9ba4-4d14c1b42c20 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:07.493236 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:07.531152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 840/1691] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:07 2022] PUT /identity/v3/projects/a8af09e38d54478397dd2f346c5d968c/users/6e8412ec926042b4ab513a82f0be8644/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:07.543326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11dfd5-9d3e-4769-a48d-5c9d37cbc7bd None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:07.545837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11dfd5-9d3e-4769-a48d-5c9d37cbc7bd None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:07.546251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11dfd5-9d3e-4769-a48d-5c9d37cbc7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:07.559341 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 852/1692] 10.222.0.70 () {58 vars in 1009 bytes} [Wed Aug 24 16:48:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:07.574975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b5cec-fd2b-4507-ab18-bfaf7af410ee None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:07.577764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-539b5cec-fd2b-4507-ab18-bfaf7af410ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:07.592843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b5cec-fd2b-4507-ab18-bfaf7af410ee None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:07.619474 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-5f11dfd5-9d3e-4769-a48d-5c9d37cbc7bd None None] MFA Rules not processed for user `6e8412ec926042b4ab513a82f0be8644`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:07.621538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-5f11dfd5-9d3e-4769-a48d-5c9d37cbc7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:07.704313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:07.724584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f373c-2660-455c-85ea-c9e1baf12b59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:07.727952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f373c-2660-455c-85ea-c9e1baf12b59 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:07.734506 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f373c-2660-455c-85ea-c9e1baf12b59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:07.734506 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f373c-2660-455c-85ea-c9e1baf12b59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:07.754188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f373c-2660-455c-85ea-c9e1baf12b59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:07.777237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f373c-2660-455c-85ea-c9e1baf12b59 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:07.835058 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 853/1693] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:48:07.836194 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f373c-2660-455c-85ea-c9e1baf12b59 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:07.847211 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5b6f373c-2660-455c-85ea-c9e1baf12b59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:07.925588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 841/1694] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:07 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:07.926590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11dfd5-9d3e-4769-a48d-5c9d37cbc7bd None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:08.113253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11dfd5-9d3e-4769-a48d-5c9d37cbc7bd None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:08.123808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df375a27-fa0d-494d-964e-1f9a103d41d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:08.124678 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df375a27-fa0d-494d-964e-1f9a103d41d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:08.125163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df375a27-fa0d-494d-964e-1f9a103d41d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:08.125289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f373c-2660-455c-85ea-c9e1baf12b59 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:08.125713 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df375a27-fa0d-494d-964e-1f9a103d41d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:08.128104 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df375a27-fa0d-494d-964e-1f9a103d41d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:08.130331 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df375a27-fa0d-494d-964e-1f9a103d41d3 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:08.139245 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74d677-3d89-4f06-aa2a-7dd714ca9535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:08.140325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74d677-3d89-4f06-aa2a-7dd714ca9535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:08.141647 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74d677-3d89-4f06-aa2a-7dd714ca9535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:08.142254 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74d677-3d89-4f06-aa2a-7dd714ca9535 admin admin] PATH_INFO: `/v3/users/c1f4a9bf7d14415e9c883fd200d47cf6` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:08.143879 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74d677-3d89-4f06-aa2a-7dd714ca9535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1f4a9bf7d14415e9c883fd200d47cf6)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:08.150943 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74d677-3d89-4f06-aa2a-7dd714ca9535 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:08.291488 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 842/1695] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:48:07 2022] POST /identity/v3/projects => generated 403 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:08.291488 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71083610-7cea-49ab-97ef-7fb99a1a100d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:08.291488 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71083610-7cea-49ab-97ef-7fb99a1a100d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:08.291488 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71083610-7cea-49ab-97ef-7fb99a1a100d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:08.330888 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-71083610-7cea-49ab-97ef-7fb99a1a100d None None] MFA Rules not processed for user `45913fc3d9ba42ea9e971db25e656c61`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:08.332219 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-71083610-7cea-49ab-97ef-7fb99a1a100d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:08.482697 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5a74d677-3d89-4f06-aa2a-7dd714ca9535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1f4a9bf7d14415e9c883fd200d47cf6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:08.484756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5a74d677-3d89-4f06-aa2a-7dd714ca9535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1f4a9bf7d14415e9c883fd200d47cf6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:08.489682 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5a74d677-3d89-4f06-aa2a-7dd714ca9535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1f4a9bf7d14415e9c883fd200d47cf6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:08.517987 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 843/1696] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:08.520552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71083610-7cea-49ab-97ef-7fb99a1a100d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:08.577661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5a74d677-3d89-4f06-aa2a-7dd714ca9535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1f4a9bf7d14415e9c883fd200d47cf6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:08.586416 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5a74d677-3d89-4f06-aa2a-7dd714ca9535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1f4a9bf7d14415e9c883fd200d47cf6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:08.607970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:08.621777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:08.625637 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 854/1697] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:48:07 2022] DELETE /identity/v3/users/c1f4a9bf7d14415e9c883fd200d47cf6 => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:08.626933 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdf3ec-5826-471f-b3fe-8193e745efd5 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:08.627230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdf3ec-5826-471f-b3fe-8193e745efd5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:08.627508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdf3ec-5826-471f-b3fe-8193e745efd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:08.657393 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71083610-7cea-49ab-97ef-7fb99a1a100d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:08.714414 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7dbdf3ec-5826-471f-b3fe-8193e745efd5 None None] MFA Rules not processed for user `7a930257a6584b04bae278aa83940264`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:08.718544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7dbdf3ec-5826-471f-b3fe-8193e745efd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:08.826322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47635f5f-a4aa-49c4-91c9-368ccd0f8e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:08.833002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47635f5f-a4aa-49c4-91c9-368ccd0f8e45 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:08.833002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47635f5f-a4aa-49c4-91c9-368ccd0f8e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:08.833002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47635f5f-a4aa-49c4-91c9-368ccd0f8e45 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:08.833002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47635f5f-a4aa-49c4-91c9-368ccd0f8e45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:08.833002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47635f5f-a4aa-49c4-91c9-368ccd0f8e45 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:08.896747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 855/1698] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:08.899681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf609bbe-00d9-4082-aa6d-894c4bbb0812 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:08.901754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf609bbe-00d9-4082-aa6d-894c4bbb0812 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:08.903634 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf609bbe-00d9-4082-aa6d-894c4bbb0812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:08.989361 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-bf609bbe-00d9-4082-aa6d-894c4bbb0812 None None] MFA Rules not processed for user `45913fc3d9ba42ea9e971db25e656c61`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:08.991636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bf609bbe-00d9-4082-aa6d-894c4bbb0812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:09.006705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 844/1699] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:48:08 2022] POST /identity/v3/users => generated 308 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:09.016728 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a087b33-59ed-4ee1-8f49-7e6c087d1575 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:09.016728 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a087b33-59ed-4ee1-8f49-7e6c087d1575 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:09.016728 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a087b33-59ed-4ee1-8f49-7e6c087d1575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:09.074165 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-6a087b33-59ed-4ee1-8f49-7e6c087d1575 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:09.074758 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6a087b33-59ed-4ee1-8f49-7e6c087d1575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:09.110384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 856/1700] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:09.111984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf609bbe-00d9-4082-aa6d-894c4bbb0812 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:09.114570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bf609bbe-00d9-4082-aa6d-894c4bbb0812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:09.115615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 845/1701] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:09.117028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49856b8-b4b6-4093-98af-e4652a4d1bd1 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:09.117313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49856b8-b4b6-4093-98af-e4652a4d1bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:09.117582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49856b8-b4b6-4093-98af-e4652a4d1bd1 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:09.118890 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 846/1702] 10.222.0.70 () {54 vars in 961 bytes} [Wed Aug 24 16:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:48:09.119834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49856b8-b4b6-4093-98af-e4652a4d1bd1 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:09.130621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e49856b8-b4b6-4093-98af-e4652a4d1bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:09.153377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49856b8-b4b6-4093-98af-e4652a4d1bd1 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:09.158699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd49a8e-23ae-4ecf-bffd-d9e7dc34bf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:09.159472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd49a8e-23ae-4ecf-bffd-d9e7dc34bf7e admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:09.160445 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd49a8e-23ae-4ecf-bffd-d9e7dc34bf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:09.160839 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd49a8e-23ae-4ecf-bffd-d9e7dc34bf7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:09.161832 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd49a8e-23ae-4ecf-bffd-d9e7dc34bf7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:09.165409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd49a8e-23ae-4ecf-bffd-d9e7dc34bf7e admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:09.229902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 847/1703] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:09.237113 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd49a8e-23ae-4ecf-bffd-d9e7dc34bf7e admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:09.245843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-9dd49a8e-23ae-4ecf-bffd-d9e7dc34bf7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:09.278699 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf609bbe-00d9-4082-aa6d-894c4bbb0812 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:09.283448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd49a8e-23ae-4ecf-bffd-d9e7dc34bf7e admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:09.296230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4adf00-ba22-4b3e-a9a8-7e8f93b51cde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:09.296954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4adf00-ba22-4b3e-a9a8-7e8f93b51cde service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:09.299141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4adf00-ba22-4b3e-a9a8-7e8f93b51cde service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:09.299439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4adf00-ba22-4b3e-a9a8-7e8f93b51cde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:09.302672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d254a47-7efd-4074-9a03-028f94dd8542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:09.304786 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d254a47-7efd-4074-9a03-028f94dd8542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:09.305244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d254a47-7efd-4074-9a03-028f94dd8542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:09.305702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d254a47-7efd-4074-9a03-028f94dd8542 admin admin] PATH_INFO: `/v3/users/f7bee141038c437eb57f3aa0e97558f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:09.318911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d254a47-7efd-4074-9a03-028f94dd8542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7bee141038c437eb57f3aa0e97558f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:09.321923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d254a47-7efd-4074-9a03-028f94dd8542 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:09.325740 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4adf00-ba22-4b3e-a9a8-7e8f93b51cde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:09.327478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4adf00-ba22-4b3e-a9a8-7e8f93b51cde service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:09.361315 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 857/1704] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:09.364374 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4adf00-ba22-4b3e-a9a8-7e8f93b51cde service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:09.378237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4b4adf00-ba22-4b3e-a9a8-7e8f93b51cde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:09.432788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4adf00-ba22-4b3e-a9a8-7e8f93b51cde service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:09.454214 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc7d08-13c8-4346-b123-15cf9a92c212 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:09.455190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc7d08-13c8-4346-b123-15cf9a92c212 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:09.455636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc7d08-13c8-4346-b123-15cf9a92c212 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:09.456055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc7d08-13c8-4346-b123-15cf9a92c212 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:09.463863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e8cc7d08-13c8-4346-b123-15cf9a92c212 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:09.488904 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc7d08-13c8-4346-b123-15cf9a92c212 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:09.492420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc7d08-13c8-4346-b123-15cf9a92c212 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:09.553397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 858/1705] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:09.554871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1e101-125e-4a86-a665-c0cb9a820bd6 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:09.555168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1e101-125e-4a86-a665-c0cb9a820bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:09.555447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1e101-125e-4a86-a665-c0cb9a820bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:09.575258 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6d254a47-7efd-4074-9a03-028f94dd8542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7bee141038c437eb57f3aa0e97558f1'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:09.576673 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6d254a47-7efd-4074-9a03-028f94dd8542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7bee141038c437eb57f3aa0e97558f1'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:09.577774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6d254a47-7efd-4074-9a03-028f94dd8542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7bee141038c437eb57f3aa0e97558f1'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:09.589485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6d254a47-7efd-4074-9a03-028f94dd8542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7bee141038c437eb57f3aa0e97558f1'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:09.633661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0ea1e101-125e-4a86-a665-c0cb9a820bd6 None None] MFA Rules not processed for user `fc4c43c85a5d49979ba803fdb1041c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:09.635522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0ea1e101-125e-4a86-a665-c0cb9a820bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:09.682760 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6d254a47-7efd-4074-9a03-028f94dd8542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7bee141038c437eb57f3aa0e97558f1'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:09.743037 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 848/1706] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:48:09 2022] DELETE /identity/v3/users/f7bee141038c437eb57f3aa0e97558f1 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:09.748678 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d254a47-7efd-4074-9a03-028f94dd8542 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:09.748678 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6d254a47-7efd-4074-9a03-028f94dd8542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:09.789047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:09.841941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:09.897004 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 859/1707] 10.222.0.70 () {58 vars in 1058 bytes} [Wed Aug 24 16:48:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:09.899721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1e101-125e-4a86-a665-c0cb9a820bd6 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:09.909962 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0ea1e101-125e-4a86-a665-c0cb9a820bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:09.932519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d254a47-7efd-4074-9a03-028f94dd8542 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:09.949287 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb1f0a-ac6f-4927-b558-1c6a8c9da6f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:09.949927 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb1f0a-ac6f-4927-b558-1c6a8c9da6f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:09.950213 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb1f0a-ac6f-4927-b558-1c6a8c9da6f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:09.950487 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb1f0a-ac6f-4927-b558-1c6a8c9da6f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:09.956237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1e101-125e-4a86-a665-c0cb9a820bd6 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:09.956914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-dddb1f0a-ac6f-4927-b558-1c6a8c9da6f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:10.144297 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360381c6-6d9d-46aa-8e1c-b980170c77d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:10.154857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360381c6-6d9d-46aa-8e1c-b980170c77d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:10.154857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360381c6-6d9d-46aa-8e1c-b980170c77d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:10.154857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360381c6-6d9d-46aa-8e1c-b980170c77d0 admin admin] PATH_INFO: `/v3/projects/2e22189dfa9d4d86a2d5d27dd7d01a63/users/3520f10e4dfb400daa0f88c2ae6a41d9/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:10.168984 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb1f0a-ac6f-4927-b558-1c6a8c9da6f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:10.174255 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb1f0a-ac6f-4927-b558-1c6a8c9da6f2 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:10.196884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360381c6-6d9d-46aa-8e1c-b980170c77d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e22189dfa9d4d86a2d5d27dd7d01a63, user_id=3520f10e4dfb400daa0f88c2ae6a41d9, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:10.208952 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 849/1708] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:10.210428 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360381c6-6d9d-46aa-8e1c-b980170c77d0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:10.211910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68e8b3-6377-441e-b47e-dc29dd359523 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:10.215735 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68e8b3-6377-441e-b47e-dc29dd359523 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:10.220620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68e8b3-6377-441e-b47e-dc29dd359523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:10.260985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-8a68e8b3-6377-441e-b47e-dc29dd359523 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:10.264039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-8a68e8b3-6377-441e-b47e-dc29dd359523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:10.280948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 850/1709] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:10.282092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68e8b3-6377-441e-b47e-dc29dd359523 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:10.307629 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 860/1710] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:09 2022] PUT /identity/v3/projects/2e22189dfa9d4d86a2d5d27dd7d01a63/users/3520f10e4dfb400daa0f88c2ae6a41d9/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:10.312335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ee4e3-2816-4b3a-91d2-d371cbad5fe4 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:10.314249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ee4e3-2816-4b3a-91d2-d371cbad5fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:10.314879 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ee4e3-2816-4b3a-91d2-d371cbad5fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:10.398594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-d56ee4e3-2816-4b3a-91d2-d371cbad5fe4 None None] MFA Rules not processed for user `88efd5eb09064aa2aa89e6d9580c6476`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:10.403077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d56ee4e3-2816-4b3a-91d2-d371cbad5fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:10.426258 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68e8b3-6377-441e-b47e-dc29dd359523 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:10.511506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e4888-9c53-4707-b3d8-77a5c8c76907 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:10.512221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e4888-9c53-4707-b3d8-77a5c8c76907 service placement] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:10.512665 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e4888-9c53-4707-b3d8-77a5c8c76907 service placement] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:10.513041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e4888-9c53-4707-b3d8-77a5c8c76907 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:10.518045 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d44e4888-9c53-4707-b3d8-77a5c8c76907 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:10.534952 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e4888-9c53-4707-b3d8-77a5c8c76907 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:10.542898 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e4888-9c53-4707-b3d8-77a5c8c76907 service placement] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:10.560464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 851/1711] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:10.560464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e4888-9c53-4707-b3d8-77a5c8c76907 service placement] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:10.560765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 861/1712] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:10.560765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ee4e3-2816-4b3a-91d2-d371cbad5fe4 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:10.572276 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d44e4888-9c53-4707-b3d8-77a5c8c76907 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:10.573540 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:10.653231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e4888-9c53-4707-b3d8-77a5c8c76907 service placement] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:10.661906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037836b-150d-4360-acb7-964946248b1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:10.662721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037836b-150d-4360-acb7-964946248b1d service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:10.663091 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037836b-150d-4360-acb7-964946248b1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:10.663481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037836b-150d-4360-acb7-964946248b1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:10.722614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ee4e3-2816-4b3a-91d2-d371cbad5fe4 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:10.733508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c0fb2-7ed7-4e69-8c23-89ab45402b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:10.734642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c0fb2-7ed7-4e69-8c23-89ab45402b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:10.735900 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c0fb2-7ed7-4e69-8c23-89ab45402b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:10.737181 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c0fb2-7ed7-4e69-8c23-89ab45402b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:10.737861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c0fb2-7ed7-4e69-8c23-89ab45402b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:10.742320 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c0fb2-7ed7-4e69-8c23-89ab45402b01 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:10.763478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 862/1713] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:10.764473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c0fb2-7ed7-4e69-8c23-89ab45402b01 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:10.771359 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-226c0fb2-7ed7-4e69-8c23-89ab45402b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:10.826604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037836b-150d-4360-acb7-964946248b1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:10.828096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037836b-150d-4360-acb7-964946248b1d service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:10.854835 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 852/1714] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:10.855692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037836b-150d-4360-acb7-964946248b1d service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:10.897310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037836b-150d-4360-acb7-964946248b1d service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:10.910134 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245349f-249f-4e4f-97c3-da36f176033b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:10.913330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245349f-249f-4e4f-97c3-da36f176033b admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:10.915433 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245349f-249f-4e4f-97c3-da36f176033b admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:10.916262 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245349f-249f-4e4f-97c3-da36f176033b admin admin] PATH_INFO: `/v3/projects/2e22189dfa9d4d86a2d5d27dd7d01a63/users/3520f10e4dfb400daa0f88c2ae6a41d9/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:10.931027 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245349f-249f-4e4f-97c3-da36f176033b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e22189dfa9d4d86a2d5d27dd7d01a63, user_id=3520f10e4dfb400daa0f88c2ae6a41d9, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:10.936423 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245349f-249f-4e4f-97c3-da36f176033b admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:10.969103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 853/1715] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:10 2022] PUT /identity/v3/projects/2e22189dfa9d4d86a2d5d27dd7d01a63/users/3520f10e4dfb400daa0f88c2ae6a41d9/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:10.972994 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10f2c8-9d3b-4b29-afdf-707ecb4dd371 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:10.973551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10f2c8-9d3b-4b29-afdf-707ecb4dd371 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:10.976318 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10f2c8-9d3b-4b29-afdf-707ecb4dd371 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:10.978485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 854/1716] 10.222.0.70 () {54 vars in 912 bytes} [Wed Aug 24 16:48:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:48:10.981904 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc769fa0-71af-47a4-98cc-b98e22c11641 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:10.983036 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc769fa0-71af-47a4-98cc-b98e22c11641 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:10.987108 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc769fa0-71af-47a4-98cc-b98e22c11641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:10.994131 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c0fb2-7ed7-4e69-8c23-89ab45402b01 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:11.031076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:11.076597 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-fc769fa0-71af-47a4-98cc-b98e22c11641 None None] MFA Rules not processed for user `3520f10e4dfb400daa0f88c2ae6a41d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:11.079563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-fc769fa0-71af-47a4-98cc-b98e22c11641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:11.218240 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a829041-e387-4cde-958b-47b1007d9e1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:11.218892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a829041-e387-4cde-958b-47b1007d9e1e service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:11.219165 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a829041-e387-4cde-958b-47b1007d9e1e service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:11.219440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a829041-e387-4cde-958b-47b1007d9e1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:11.433859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 855/1717] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:10 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:11.433859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10e0b0-fdd5-4683-a6f3-0d45764478c6 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:11.433859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10e0b0-fdd5-4683-a6f3-0d45764478c6 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:11.433859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10e0b0-fdd5-4683-a6f3-0d45764478c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:11.433859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-6c10e0b0-fdd5-4683-a6f3-0d45764478c6 None None] MFA Rules not processed for user `942b54c39ae5403c8ae3bcca1c31bf1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:11.433859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6c10e0b0-fdd5-4683-a6f3-0d45764478c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:11.435482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a829041-e387-4cde-958b-47b1007d9e1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:11.435482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a829041-e387-4cde-958b-47b1007d9e1e service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:11.435482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 863/1718] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:11.435482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a829041-e387-4cde-958b-47b1007d9e1e service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:11.435482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7a829041-e387-4cde-958b-47b1007d9e1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:11.504258 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 856/1719] 10.222.0.70 () {58 vars in 1009 bytes} [Wed Aug 24 16:48:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:11.506111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10e0b0-fdd5-4683-a6f3-0d45764478c6 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:11.510286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6c10e0b0-fdd5-4683-a6f3-0d45764478c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:11.521144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a829041-e387-4cde-958b-47b1007d9e1e service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:11.529774 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42880396-681e-4381-bb94-6ca4db1043fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:11.530403 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42880396-681e-4381-bb94-6ca4db1043fb service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:11.530687 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42880396-681e-4381-bb94-6ca4db1043fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:11.530964 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42880396-681e-4381-bb94-6ca4db1043fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:11.558060 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42880396-681e-4381-bb94-6ca4db1043fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:11.559486 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42880396-681e-4381-bb94-6ca4db1043fb service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:11.584757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 864/1720] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:11 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:11.600871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42880396-681e-4381-bb94-6ca4db1043fb service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:11.617795 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42880396-681e-4381-bb94-6ca4db1043fb service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:11.627489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ba564-f66d-41aa-a171-74bba8d2c4df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:11.628251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ba564-f66d-41aa-a171-74bba8d2c4df service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:11.628575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ba564-f66d-41aa-a171-74bba8d2c4df service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:11.628848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ba564-f66d-41aa-a171-74bba8d2c4df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:11.637489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ba564-f66d-41aa-a171-74bba8d2c4df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:11.638939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ba564-f66d-41aa-a171-74bba8d2c4df service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:11.656047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 865/1721] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:11.667456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10e0b0-fdd5-4683-a6f3-0d45764478c6 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:11.673962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded270c9-c63e-43ee-b405-5190e1149c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:11.677208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded270c9-c63e-43ee-b405-5190e1149c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:11.677208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded270c9-c63e-43ee-b405-5190e1149c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:11.677208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded270c9-c63e-43ee-b405-5190e1149c21 admin admin] PATH_INFO: `/v3/projects/3af266d023b7412c8b878b15b5fff053` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:11.678644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded270c9-c63e-43ee-b405-5190e1149c21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3af266d023b7412c8b878b15b5fff053)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:11.684964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded270c9-c63e-43ee-b405-5190e1149c21 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:11.840222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ded270c9-c63e-43ee-b405-5190e1149c21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3af266d023b7412c8b878b15b5fff053', 'request_id': 'req-ded270c9-c63e-43ee-b405-5190e1149c21'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:11.861040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ded270c9-c63e-43ee-b405-5190e1149c21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3af266d023b7412c8b878b15b5fff053', 'request_id': 'req-ded270c9-c63e-43ee-b405-5190e1149c21'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:11.870592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ded270c9-c63e-43ee-b405-5190e1149c21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3af266d023b7412c8b878b15b5fff053', 'request_id': 'req-ded270c9-c63e-43ee-b405-5190e1149c21'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:11.961693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ba564-f66d-41aa-a171-74bba8d2c4df service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:11.961834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ded270c9-c63e-43ee-b405-5190e1149c21 admin admin] The token cache is being invalidate because project 3af266d023b7412c8b878b15b5fff053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:48:11.962187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ded270c9-c63e-43ee-b405-5190e1149c21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:11.963435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ded270c9-c63e-43ee-b405-5190e1149c21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:11.968725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 857/1722] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:48:11 2022] DELETE /identity/v3/projects/3af266d023b7412c8b878b15b5fff053 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:11.969775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded270c9-c63e-43ee-b405-5190e1149c21 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:11.976360 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ded270c9-c63e-43ee-b405-5190e1149c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:11.990597 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ba564-f66d-41aa-a171-74bba8d2c4df service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:12.045232 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:12.114522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:12.146964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded270c9-c63e-43ee-b405-5190e1149c21 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:12.157584 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907802f5-865a-4ea0-9041-97c50b32c58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:12.174023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8f6db-4674-480f-a31f-cc0dc83c86cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:12.174467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907802f5-865a-4ea0-9041-97c50b32c58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:12.174467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907802f5-865a-4ea0-9041-97c50b32c58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:12.174467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907802f5-865a-4ea0-9041-97c50b32c58d admin admin] PATH_INFO: `/v3/users/ea19f29fa4ae42b297a29e90484bc365` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:12.174728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8f6db-4674-480f-a31f-cc0dc83c86cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:12.174728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8f6db-4674-480f-a31f-cc0dc83c86cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:12.174728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8f6db-4674-480f-a31f-cc0dc83c86cf admin admin] PATH_INFO: `/v3/users/6e8412ec926042b4ab513a82f0be8644` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:12.174962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907802f5-865a-4ea0-9041-97c50b32c58d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea19f29fa4ae42b297a29e90484bc365)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:12.174962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907802f5-865a-4ea0-9041-97c50b32c58d admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:12.175127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8f6db-4674-480f-a31f-cc0dc83c86cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8412ec926042b4ab513a82f0be8644)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:12.175127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8f6db-4674-480f-a31f-cc0dc83c86cf admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:12.450036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-09a8f6db-4674-480f-a31f-cc0dc83c86cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8412ec926042b4ab513a82f0be8644'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:12.451456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-09a8f6db-4674-480f-a31f-cc0dc83c86cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8412ec926042b4ab513a82f0be8644'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:12.452090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-09a8f6db-4674-480f-a31f-cc0dc83c86cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8412ec926042b4ab513a82f0be8644'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:12.460633 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-907802f5-865a-4ea0-9041-97c50b32c58d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea19f29fa4ae42b297a29e90484bc365'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:12.460633 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-907802f5-865a-4ea0-9041-97c50b32c58d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea19f29fa4ae42b297a29e90484bc365'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:12.461347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-907802f5-865a-4ea0-9041-97c50b32c58d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea19f29fa4ae42b297a29e90484bc365'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:12.484060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-907802f5-865a-4ea0-9041-97c50b32c58d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea19f29fa4ae42b297a29e90484bc365'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:12.591002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-09a8f6db-4674-480f-a31f-cc0dc83c86cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8412ec926042b4ab513a82f0be8644'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:12.602300 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-09a8f6db-4674-480f-a31f-cc0dc83c86cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8412ec926042b4ab513a82f0be8644'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:12.632928 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 866/1723] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:48:11 2022] DELETE /identity/v3/users/6e8412ec926042b4ab513a82f0be8644 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:48:12.634088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8f6db-4674-480f-a31f-cc0dc83c86cf admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:12.636382 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-09a8f6db-4674-480f-a31f-cc0dc83c86cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:12.678937 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-907802f5-865a-4ea0-9041-97c50b32c58d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea19f29fa4ae42b297a29e90484bc365'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:12.731894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 858/1724] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:48:11 2022] DELETE /identity/v3/users/ea19f29fa4ae42b297a29e90484bc365 => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:12.731894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7489b02-e313-4460-9fa3-71b1833932b8 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:12.731894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7489b02-e313-4460-9fa3-71b1833932b8 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:12.731894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7489b02-e313-4460-9fa3-71b1833932b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:12.780600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8f6db-4674-480f-a31f-cc0dc83c86cf admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:12.784304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-e7489b02-e313-4460-9fa3-71b1833932b8 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:12.787025 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e7489b02-e313-4460-9fa3-71b1833932b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:12.957381 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d20c93-4a79-44c3-a022-bdd01f68a84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:12.957980 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d20c93-4a79-44c3-a022-bdd01f68a84d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:12.958469 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d20c93-4a79-44c3-a022-bdd01f68a84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:12.958745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d20c93-4a79-44c3-a022-bdd01f68a84d admin admin] PATH_INFO: `/v3/projects/260e43af05a741139d93472a06effdd5` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:12.963414 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d20c93-4a79-44c3-a022-bdd01f68a84d admin admin] RBAC: Authorizing `identity:delete_project(project_id=260e43af05a741139d93472a06effdd5)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:12.972030 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d20c93-4a79-44c3-a022-bdd01f68a84d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:12.990836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 859/1725] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:13.030165 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:13.050072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7489b02-e313-4460-9fa3-71b1833932b8 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:13.051504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e7489b02-e313-4460-9fa3-71b1833932b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:13.165329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b2d20c93-4a79-44c3-a022-bdd01f68a84d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '260e43af05a741139d93472a06effdd5', 'request_id': 'req-b2d20c93-4a79-44c3-a022-bdd01f68a84d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:13.197295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b2d20c93-4a79-44c3-a022-bdd01f68a84d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '260e43af05a741139d93472a06effdd5', 'request_id': 'req-b2d20c93-4a79-44c3-a022-bdd01f68a84d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:13.209058 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b2d20c93-4a79-44c3-a022-bdd01f68a84d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '260e43af05a741139d93472a06effdd5', 'request_id': 'req-b2d20c93-4a79-44c3-a022-bdd01f68a84d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:13.252148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7489b02-e313-4460-9fa3-71b1833932b8 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:13.360661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b2d20c93-4a79-44c3-a022-bdd01f68a84d admin admin] The token cache is being invalidate because project 260e43af05a741139d93472a06effdd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:48:13.364536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b2d20c93-4a79-44c3-a022-bdd01f68a84d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:13.364536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b2d20c93-4a79-44c3-a022-bdd01f68a84d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:13.364536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 867/1726] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:48:12 2022] DELETE /identity/v3/projects/260e43af05a741139d93472a06effdd5 => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:13.365216 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2d410-f741-496c-84f3-b537357128ad None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:13.365503 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2d410-f741-496c-84f3-b537357128ad None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:13.365776 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2d410-f741-496c-84f3-b537357128ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:13.393221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a732e-3133-4343-a996-02177eb62ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:13.393912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a732e-3133-4343-a996-02177eb62ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:13.394208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a732e-3133-4343-a996-02177eb62ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:13.394518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a732e-3133-4343-a996-02177eb62ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:13.399789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ae3a732e-3133-4343-a996-02177eb62ac7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:13.439295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-82e2d410-f741-496c-84f3-b537357128ad None None] MFA Rules not processed for user `45913fc3d9ba42ea9e971db25e656c61`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:13.441512 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-82e2d410-f741-496c-84f3-b537357128ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:13.598953 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 868/1727] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:13.604058 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2d410-f741-496c-84f3-b537357128ad None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:13.607265 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-82e2d410-f741-496c-84f3-b537357128ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:13.638878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a732e-3133-4343-a996-02177eb62ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:13.646011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a732e-3133-4343-a996-02177eb62ac7 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:13.650422 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2d410-f741-496c-84f3-b537357128ad None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:13.670868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ca9c4-6e9d-4df2-bb43-0a18f4eceb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:13.674436 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ca9c4-6e9d-4df2-bb43-0a18f4eceb0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:13.675098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ca9c4-6e9d-4df2-bb43-0a18f4eceb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:13.676150 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ca9c4-6e9d-4df2-bb43-0a18f4eceb0f admin admin] PATH_INFO: `/v3/users/bbd5b43e29af4a4585bcd4eceef56a12` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:13.680019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ca9c4-6e9d-4df2-bb43-0a18f4eceb0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbd5b43e29af4a4585bcd4eceef56a12)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:13.682612 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ca9c4-6e9d-4df2-bb43-0a18f4eceb0f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:13.684044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 860/1728] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:48:13.814565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:14.077656 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f94ca9c4-6e9d-4df2-bb43-0a18f4eceb0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbd5b43e29af4a4585bcd4eceef56a12'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:14.081408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f94ca9c4-6e9d-4df2-bb43-0a18f4eceb0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbd5b43e29af4a4585bcd4eceef56a12'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:14.083389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f94ca9c4-6e9d-4df2-bb43-0a18f4eceb0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbd5b43e29af4a4585bcd4eceef56a12'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:14.231280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f94ca9c4-6e9d-4df2-bb43-0a18f4eceb0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbd5b43e29af4a4585bcd4eceef56a12'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:14.241662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f94ca9c4-6e9d-4df2-bb43-0a18f4eceb0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbd5b43e29af4a4585bcd4eceef56a12'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:14.296840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 869/1729] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:48:13 2022] DELETE /identity/v3/users/bbd5b43e29af4a4585bcd4eceef56a12 => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:14.405400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a732e-3133-4343-a996-02177eb62ac7 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:14.407136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ae3a732e-3133-4343-a996-02177eb62ac7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:14.443510 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a732e-3133-4343-a996-02177eb62ac7 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:14.628997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6d1fa-453a-4676-ae47-f99816009abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:14.629947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6d1fa-453a-4676-ae47-f99816009abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:14.630442 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6d1fa-453a-4676-ae47-f99816009abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:14.632207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6d1fa-453a-4676-ae47-f99816009abd admin admin] PATH_INFO: `/v3/projects/fd95ac269f3442308000bc994cacfe4e` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:14.634992 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6d1fa-453a-4676-ae47-f99816009abd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd95ac269f3442308000bc994cacfe4e)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:14.641912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6d1fa-453a-4676-ae47-f99816009abd admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:14.888686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c0b6d1fa-453a-4676-ae47-f99816009abd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd95ac269f3442308000bc994cacfe4e', 'request_id': 'req-c0b6d1fa-453a-4676-ae47-f99816009abd'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:14.897622 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c0b6d1fa-453a-4676-ae47-f99816009abd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd95ac269f3442308000bc994cacfe4e', 'request_id': 'req-c0b6d1fa-453a-4676-ae47-f99816009abd'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:14.907910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c0b6d1fa-453a-4676-ae47-f99816009abd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd95ac269f3442308000bc994cacfe4e', 'request_id': 'req-c0b6d1fa-453a-4676-ae47-f99816009abd'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:15.013523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c0b6d1fa-453a-4676-ae47-f99816009abd admin admin] The token cache is being invalidate because project fd95ac269f3442308000bc994cacfe4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:48:15.013523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c0b6d1fa-453a-4676-ae47-f99816009abd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:15.014313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c0b6d1fa-453a-4676-ae47-f99816009abd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:15.016887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 861/1730] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:48:14 2022] DELETE /identity/v3/projects/fd95ac269f3442308000bc994cacfe4e => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:15.022024 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cead1ef-6b85-460a-91cd-8c97013688f3 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:15.023016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cead1ef-6b85-460a-91cd-8c97013688f3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:15.023458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cead1ef-6b85-460a-91cd-8c97013688f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:15.073708 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0cead1ef-6b85-460a-91cd-8c97013688f3 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:15.082290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0cead1ef-6b85-460a-91cd-8c97013688f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:15.195031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6d1fa-453a-4676-ae47-f99816009abd admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:15.196909 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c0b6d1fa-453a-4676-ae47-f99816009abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:15.255153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 870/1731] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:15.264442 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fb62e-2496-4519-b200-28cfffe14421 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:15.264976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fb62e-2496-4519-b200-28cfffe14421 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:15.265397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fb62e-2496-4519-b200-28cfffe14421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:15.310183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-837fb62e-2496-4519-b200-28cfffe14421 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:15.312545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-837fb62e-2496-4519-b200-28cfffe14421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:15.337138 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 871/1732] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:15.342393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837fb62e-2496-4519-b200-28cfffe14421 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:15.355346 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6d1fa-453a-4676-ae47-f99816009abd admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:15.366662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837fb62e-2496-4519-b200-28cfffe14421 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:15.367448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc88c76-2300-4b0e-85ff-aabbe648a940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:15.368197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc88c76-2300-4b0e-85ff-aabbe648a940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:15.368366 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc88c76-2300-4b0e-85ff-aabbe648a940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:15.369707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc88c76-2300-4b0e-85ff-aabbe648a940 admin admin] PATH_INFO: `/v3/projects/a8af09e38d54478397dd2f346c5d968c` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:15.371894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc88c76-2300-4b0e-85ff-aabbe648a940 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8af09e38d54478397dd2f346c5d968c)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:15.375188 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc88c76-2300-4b0e-85ff-aabbe648a940 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:15.384809 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2f4d6-77bf-4196-9728-69a89b4c8bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:15.385792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2f4d6-77bf-4196-9728-69a89b4c8bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:15.386079 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2f4d6-77bf-4196-9728-69a89b4c8bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:15.386350 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2f4d6-77bf-4196-9728-69a89b4c8bf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:15.388584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2f4d6-77bf-4196-9728-69a89b4c8bf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:48:15.389477 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2f4d6-77bf-4196-9728-69a89b4c8bf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:15.392908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2f4d6-77bf-4196-9728-69a89b4c8bf8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:15.428594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 872/1733] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:48:15.433585 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2f4d6-77bf-4196-9728-69a89b4c8bf8 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:15.466518 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2f4d6-77bf-4196-9728-69a89b4c8bf8 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:15.492869 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346b745-2fdd-416c-b768-74fa7617bfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:15.493536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346b745-2fdd-416c-b768-74fa7617bfcb admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:15.493823 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346b745-2fdd-416c-b768-74fa7617bfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:15.498537 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346b745-2fdd-416c-b768-74fa7617bfcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:15.500193 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346b745-2fdd-416c-b768-74fa7617bfcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:15.505502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346b745-2fdd-416c-b768-74fa7617bfcb admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:15.610439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-afc88c76-2300-4b0e-85ff-aabbe648a940 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8af09e38d54478397dd2f346c5d968c', 'request_id': 'req-afc88c76-2300-4b0e-85ff-aabbe648a940'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:15.631948 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 873/1734] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:48:15 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:15.641158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-afc88c76-2300-4b0e-85ff-aabbe648a940 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8af09e38d54478397dd2f346c5d968c', 'request_id': 'req-afc88c76-2300-4b0e-85ff-aabbe648a940'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:15.644190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346b745-2fdd-416c-b768-74fa7617bfcb admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:15.649162 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-afc88c76-2300-4b0e-85ff-aabbe648a940 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8af09e38d54478397dd2f346c5d968c', 'request_id': 'req-afc88c76-2300-4b0e-85ff-aabbe648a940'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:15.651430 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c346b745-2fdd-416c-b768-74fa7617bfcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:15.751842 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-afc88c76-2300-4b0e-85ff-aabbe648a940 admin admin] The token cache is being invalidate because project a8af09e38d54478397dd2f346c5d968c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:48:15.752598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-afc88c76-2300-4b0e-85ff-aabbe648a940 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:15.765556 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-afc88c76-2300-4b0e-85ff-aabbe648a940 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:15.778250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 862/1735] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:48:15 2022] DELETE /identity/v3/projects/a8af09e38d54478397dd2f346c5d968c => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:48:15.778250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc88c76-2300-4b0e-85ff-aabbe648a940 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:15.781127 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-afc88c76-2300-4b0e-85ff-aabbe648a940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:15.819677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346b745-2fdd-416c-b768-74fa7617bfcb admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:15.845124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c5a80-0fed-43c0-99be-435f7c4d7202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:15.846116 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c5a80-0fed-43c0-99be-435f7c4d7202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:15.846903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c5a80-0fed-43c0-99be-435f7c4d7202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:15.847364 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c5a80-0fed-43c0-99be-435f7c4d7202 admin admin] PATH_INFO: `/v3/projects/91289101ca454ee0ba68947a2a96be54` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:15.850085 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c5a80-0fed-43c0-99be-435f7c4d7202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91289101ca454ee0ba68947a2a96be54)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:15.856881 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c5a80-0fed-43c0-99be-435f7c4d7202 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:15.953992 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:15.981387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc88c76-2300-4b0e-85ff-aabbe648a940 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:16.012583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1feddd5-5b8f-489a-a911-1f30f3698fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:16.019560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1feddd5-5b8f-489a-a911-1f30f3698fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:16.019560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1feddd5-5b8f-489a-a911-1f30f3698fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:16.019560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1feddd5-5b8f-489a-a911-1f30f3698fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:16.019560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1feddd5-5b8f-489a-a911-1f30f3698fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:16.020307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1feddd5-5b8f-489a-a911-1f30f3698fc5 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:16.109337 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f89c5a80-0fed-43c0-99be-435f7c4d7202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91289101ca454ee0ba68947a2a96be54', 'request_id': 'req-f89c5a80-0fed-43c0-99be-435f7c4d7202'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:16.128976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f89c5a80-0fed-43c0-99be-435f7c4d7202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91289101ca454ee0ba68947a2a96be54', 'request_id': 'req-f89c5a80-0fed-43c0-99be-435f7c4d7202'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:16.149692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f89c5a80-0fed-43c0-99be-435f7c4d7202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91289101ca454ee0ba68947a2a96be54', 'request_id': 'req-f89c5a80-0fed-43c0-99be-435f7c4d7202'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:16.156361 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:16.198681 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 863/1736] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:48:15 2022] POST /identity/v3/users => generated 293 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:16.199690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1feddd5-5b8f-489a-a911-1f30f3698fc5 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:16.206275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-f1feddd5-5b8f-489a-a911-1f30f3698fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:16.241183 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1feddd5-5b8f-489a-a911-1f30f3698fc5 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:16.296172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f89c5a80-0fed-43c0-99be-435f7c4d7202 admin admin] The token cache is being invalidate because project 91289101ca454ee0ba68947a2a96be54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:48:16.296743 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f89c5a80-0fed-43c0-99be-435f7c4d7202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:16.301108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f89c5a80-0fed-43c0-99be-435f7c4d7202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:16.305796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 874/1737] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:48:15 2022] DELETE /identity/v3/projects/91289101ca454ee0ba68947a2a96be54 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:16.306890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c5a80-0fed-43c0-99be-435f7c4d7202 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:16.310228 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f89c5a80-0fed-43c0-99be-435f7c4d7202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:16.336264 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7bb33-a99c-4c0e-bce0-08b782d55ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:16.338687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7bb33-a99c-4c0e-bce0-08b782d55ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:16.339069 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7bb33-a99c-4c0e-bce0-08b782d55ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:16.339439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7bb33-a99c-4c0e-bce0-08b782d55ce3 admin admin] PATH_INFO: `/v3/users/3520f10e4dfb400daa0f88c2ae6a41d9` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:16.340785 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7bb33-a99c-4c0e-bce0-08b782d55ce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3520f10e4dfb400daa0f88c2ae6a41d9)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:16.343007 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7bb33-a99c-4c0e-bce0-08b782d55ce3 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:16.458814 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c5a80-0fed-43c0-99be-435f7c4d7202 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:16.465136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e762587-3bca-45d3-865f-5f6d6765f7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:16.466415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e762587-3bca-45d3-865f-5f6d6765f7d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:16.466877 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e762587-3bca-45d3-865f-5f6d6765f7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:16.467167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e762587-3bca-45d3-865f-5f6d6765f7d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:16.467834 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e762587-3bca-45d3-865f-5f6d6765f7d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:16.469983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e762587-3bca-45d3-865f-5f6d6765f7d3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:16.483588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 875/1738] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:16.485782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707895e-3bfb-47fd-b8e0-a8a8c41e093c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:16.486148 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707895e-3bfb-47fd-b8e0-a8a8c41e093c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:16.486952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707895e-3bfb-47fd-b8e0-a8a8c41e093c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:16.514518 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4707895e-3bfb-47fd-b8e0-a8a8c41e093c None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:16.515766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4707895e-3bfb-47fd-b8e0-a8a8c41e093c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:16.526060 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 876/1739] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:16.526982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707895e-3bfb-47fd-b8e0-a8a8c41e093c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:16.538919 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707895e-3bfb-47fd-b8e0-a8a8c41e093c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:16.546453 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10eb31-57f0-4ba6-8855-7aeef735d35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:16.547083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10eb31-57f0-4ba6-8855-7aeef735d35a admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:16.551346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10eb31-57f0-4ba6-8855-7aeef735d35a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:16.551346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10eb31-57f0-4ba6-8855-7aeef735d35a admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:16.551346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10eb31-57f0-4ba6-8855-7aeef735d35a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:16.551346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10eb31-57f0-4ba6-8855-7aeef735d35a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:16.565791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 877/1740] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:16.567144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2c1c0-8c27-4573-9c61-783c5ecf43b6 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:16.567440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2c1c0-8c27-4573-9c61-783c5ecf43b6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:16.567723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2c1c0-8c27-4573-9c61-783c5ecf43b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:16.632552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-82f2c1c0-8c27-4573-9c61-783c5ecf43b6 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:16.633789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-82f2c1c0-8c27-4573-9c61-783c5ecf43b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:16.653914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 878/1741] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:16.654827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2c1c0-8c27-4573-9c61-783c5ecf43b6 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:16.670102 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2c1c0-8c27-4573-9c61-783c5ecf43b6 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:16.672582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d6d7bb33-a99c-4c0e-bce0-08b782d55ce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3520f10e4dfb400daa0f88c2ae6a41d9'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:16.674517 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d6d7bb33-a99c-4c0e-bce0-08b782d55ce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3520f10e4dfb400daa0f88c2ae6a41d9'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:16.675810 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d6d7bb33-a99c-4c0e-bce0-08b782d55ce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3520f10e4dfb400daa0f88c2ae6a41d9'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:16.685384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218ebd5-56dd-46d0-b955-d0383ce83431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:16.686079 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218ebd5-56dd-46d0-b955-d0383ce83431 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:16.686888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218ebd5-56dd-46d0-b955-d0383ce83431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:16.687179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218ebd5-56dd-46d0-b955-d0383ce83431 admin admin] PATH_INFO: `/v3/projects/67baf73b311f4f39a880a8651d690758/users/f89e108317d04b06835388e315172037/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:16.698325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d6d7bb33-a99c-4c0e-bce0-08b782d55ce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3520f10e4dfb400daa0f88c2ae6a41d9'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:16.723920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218ebd5-56dd-46d0-b955-d0383ce83431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67baf73b311f4f39a880a8651d690758, user_id=f89e108317d04b06835388e315172037, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:16.726747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218ebd5-56dd-46d0-b955-d0383ce83431 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:16.819437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d6d7bb33-a99c-4c0e-bce0-08b782d55ce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3520f10e4dfb400daa0f88c2ae6a41d9'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:16.851307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 879/1742] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:16 2022] PUT /identity/v3/projects/67baf73b311f4f39a880a8651d690758/users/f89e108317d04b06835388e315172037/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:16.852731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b9193-cb99-4bdc-95a6-7bce7e02725c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:16.853697 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b9193-cb99-4bdc-95a6-7bce7e02725c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:16.853983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b9193-cb99-4bdc-95a6-7bce7e02725c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:16.887643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 864/1743] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:48:16 2022] DELETE /identity/v3/users/3520f10e4dfb400daa0f88c2ae6a41d9 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:16.888328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7bb33-a99c-4c0e-bce0-08b782d55ce3 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:16.893448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d6d7bb33-a99c-4c0e-bce0-08b782d55ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:16.965683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a05b9193-cb99-4bdc-95a6-7bce7e02725c None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:16.965925 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a05b9193-cb99-4bdc-95a6-7bce7e02725c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:17.122977 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 880/1744] 10.222.0.70 () {58 vars in 1009 bytes} [Wed Aug 24 16:48:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:17.124346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e8fff-c949-4972-84f8-99231032ba49 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:17.124666 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e8fff-c949-4972-84f8-99231032ba49 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:17.124945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e8fff-c949-4972-84f8-99231032ba49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:17.126205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7bb33-a99c-4c0e-bce0-08b782d55ce3 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:17.132013 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4a3f9-e3cf-41a1-9588-8e9e98aea008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:17.132883 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4a3f9-e3cf-41a1-9588-8e9e98aea008 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:17.132983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4a3f9-e3cf-41a1-9588-8e9e98aea008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:17.133216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4a3f9-e3cf-41a1-9588-8e9e98aea008 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:17.133868 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4a3f9-e3cf-41a1-9588-8e9e98aea008 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:48:17.134265 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4a3f9-e3cf-41a1-9588-8e9e98aea008 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:17.135970 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4a3f9-e3cf-41a1-9588-8e9e98aea008 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:17.152098 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 865/1745] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:48:17.153503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2555bf7-c873-41a7-9a2c-5e552ebe12c2 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:17.153791 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2555bf7-c873-41a7-9a2c-5e552ebe12c2 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:17.154072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2555bf7-c873-41a7-9a2c-5e552ebe12c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:17.167997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-687e8fff-c949-4972-84f8-99231032ba49 None None] MFA Rules not processed for user `7a930257a6584b04bae278aa83940264`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:17.169291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-687e8fff-c949-4972-84f8-99231032ba49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:17.194730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-b2555bf7-c873-41a7-9a2c-5e552ebe12c2 None None] MFA Rules not processed for user `f89e108317d04b06835388e315172037`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:17.196791 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b2555bf7-c873-41a7-9a2c-5e552ebe12c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:17.288221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 866/1746] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:17.303577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2555bf7-c873-41a7-9a2c-5e552ebe12c2 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:17.337815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2555bf7-c873-41a7-9a2c-5e552ebe12c2 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:17.345124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce856d-d2a8-4036-8346-59959c72489c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:17.347472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce856d-d2a8-4036-8346-59959c72489c admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:17.348151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce856d-d2a8-4036-8346-59959c72489c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:17.349144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce856d-d2a8-4036-8346-59959c72489c admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:17.352028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce856d-d2a8-4036-8346-59959c72489c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:17.354294 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 881/1747] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:17.362549 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e8fff-c949-4972-84f8-99231032ba49 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:17.368250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-687e8fff-c949-4972-84f8-99231032ba49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:17.386441 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce856d-d2a8-4036-8346-59959c72489c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:17.555487 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 867/1748] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:48:17 2022] POST /identity/v3/projects => generated 367 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:17.558442 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce856d-d2a8-4036-8346-59959c72489c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:17.683023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e8fff-c949-4972-84f8-99231032ba49 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:17.769415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce856d-d2a8-4036-8346-59959c72489c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:17.791862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9f969-e70f-4449-a7e6-884404538f39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:17.792193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631eaf6-0597-43f7-8615-80d037c64267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:17.792341 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9f969-e70f-4449-a7e6-884404538f39 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:17.792747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9f969-e70f-4449-a7e6-884404538f39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:17.793256 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9f969-e70f-4449-a7e6-884404538f39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:17.797295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631eaf6-0597-43f7-8615-80d037c64267 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:17.797608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631eaf6-0597-43f7-8615-80d037c64267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:17.797853 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631eaf6-0597-43f7-8615-80d037c64267 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:17.798512 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631eaf6-0597-43f7-8615-80d037c64267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:17.810330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631eaf6-0597-43f7-8615-80d037c64267 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:17.926647 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9f969-e70f-4449-a7e6-884404538f39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:17.928524 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9f969-e70f-4449-a7e6-884404538f39 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:17.955315 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 882/1749] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:17.956438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9f969-e70f-4449-a7e6-884404538f39 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:17.958986 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-76b9f969-e70f-4449-a7e6-884404538f39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:17.987673 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 868/1750] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:48:17 2022] POST /identity/v3/users => generated 284 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:17.987909 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f8642-a418-4028-bea8-7999e4a0814a None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:17.988196 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f8642-a418-4028-bea8-7999e4a0814a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:17.988491 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f8642-a418-4028-bea8-7999e4a0814a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:18.063018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-670f8642-a418-4028-bea8-7999e4a0814a None None] MFA Rules not processed for user `88efd5eb09064aa2aa89e6d9580c6476`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:18.074646 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-670f8642-a418-4028-bea8-7999e4a0814a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:18.085889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9f969-e70f-4449-a7e6-884404538f39 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:18.111645 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fc139-6e54-4382-9812-a6ee0771e1f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:18.112561 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fc139-6e54-4382-9812-a6ee0771e1f3 service swift] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:18.113752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fc139-6e54-4382-9812-a6ee0771e1f3 service swift] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:18.114389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fc139-6e54-4382-9812-a6ee0771e1f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:18.197049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 869/1751] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:18.198769 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f8642-a418-4028-bea8-7999e4a0814a None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:18.211608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f8642-a418-4028-bea8-7999e4a0814a None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:18.216838 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418d198-38d0-408a-94d0-ceb295c44e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:18.217781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418d198-38d0-408a-94d0-ceb295c44e22 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:18.218300 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418d198-38d0-408a-94d0-ceb295c44e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:18.218998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418d198-38d0-408a-94d0-ceb295c44e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:18.220489 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418d198-38d0-408a-94d0-ceb295c44e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:18.222615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418d198-38d0-408a-94d0-ceb295c44e22 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:18.238748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 870/1752] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:18.238748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418d198-38d0-408a-94d0-ceb295c44e22 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:18.238748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c418d198-38d0-408a-94d0-ceb295c44e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:18.255620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418d198-38d0-408a-94d0-ceb295c44e22 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:18.260448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268106ef-986d-449a-8405-2c2454b1350f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:18.269691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268106ef-986d-449a-8405-2c2454b1350f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:18.269691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268106ef-986d-449a-8405-2c2454b1350f admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:18.269691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268106ef-986d-449a-8405-2c2454b1350f admin admin] PATH_INFO: `/v3/users/f89e108317d04b06835388e315172037` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:18.269691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268106ef-986d-449a-8405-2c2454b1350f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f89e108317d04b06835388e315172037)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:18.270485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268106ef-986d-449a-8405-2c2454b1350f admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:18.310437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1fc139-6e54-4382-9812-a6ee0771e1f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:18.312073 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1fc139-6e54-4382-9812-a6ee0771e1f3 service swift] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:18.334949 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 883/1753] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 16:48:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:48:18.335983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fc139-6e54-4382-9812-a6ee0771e1f3 service swift] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:18.365284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fc139-6e54-4382-9812-a6ee0771e1f3 service swift] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:18.384531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d821b0a-ed2c-497e-92f9-e66631942e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:18.384531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d821b0a-ed2c-497e-92f9-e66631942e9a admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:18.384531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d821b0a-ed2c-497e-92f9-e66631942e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:18.384531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d821b0a-ed2c-497e-92f9-e66631942e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:18.385038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d821b0a-ed2c-497e-92f9-e66631942e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:18.387682 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d821b0a-ed2c-497e-92f9-e66631942e9a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:18.413865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 884/1754] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:18.416347 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d821b0a-ed2c-497e-92f9-e66631942e9a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:18.418161 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-1d821b0a-ed2c-497e-92f9-e66631942e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:18.432781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d821b0a-ed2c-497e-92f9-e66631942e9a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:18.439812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ac711-586b-43b3-b2a3-0b507a008811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:18.440558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ac711-586b-43b3-b2a3-0b507a008811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:18.440848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ac711-586b-43b3-b2a3-0b507a008811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:18.441130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ac711-586b-43b3-b2a3-0b507a008811 admin admin] PATH_INFO: `/v3/projects/2e22189dfa9d4d86a2d5d27dd7d01a63` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:18.442391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ac711-586b-43b3-b2a3-0b507a008811 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e22189dfa9d4d86a2d5d27dd7d01a63)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:18.444730 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ac711-586b-43b3-b2a3-0b507a008811 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:18.545870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-268106ef-986d-449a-8405-2c2454b1350f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f89e108317d04b06835388e315172037'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:18.547189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-268106ef-986d-449a-8405-2c2454b1350f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f89e108317d04b06835388e315172037'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:18.548659 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-268106ef-986d-449a-8405-2c2454b1350f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f89e108317d04b06835388e315172037'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:18.560774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-268106ef-986d-449a-8405-2c2454b1350f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f89e108317d04b06835388e315172037'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:18.617921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-887ac711-586b-43b3-b2a3-0b507a008811 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e22189dfa9d4d86a2d5d27dd7d01a63', 'request_id': 'req-887ac711-586b-43b3-b2a3-0b507a008811'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:18.638165 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-887ac711-586b-43b3-b2a3-0b507a008811 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e22189dfa9d4d86a2d5d27dd7d01a63', 'request_id': 'req-887ac711-586b-43b3-b2a3-0b507a008811'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:18.667508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-887ac711-586b-43b3-b2a3-0b507a008811 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e22189dfa9d4d86a2d5d27dd7d01a63', 'request_id': 'req-887ac711-586b-43b3-b2a3-0b507a008811'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:18.683762 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-268106ef-986d-449a-8405-2c2454b1350f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f89e108317d04b06835388e315172037'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:18.756098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-887ac711-586b-43b3-b2a3-0b507a008811 admin admin] The token cache is being invalidate because project 2e22189dfa9d4d86a2d5d27dd7d01a63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:48:18.756521 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-887ac711-586b-43b3-b2a3-0b507a008811 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:18.758089 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-887ac711-586b-43b3-b2a3-0b507a008811 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:18.762052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 885/1755] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:48:18 2022] DELETE /identity/v3/projects/2e22189dfa9d4d86a2d5d27dd7d01a63 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:18.766732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ac711-586b-43b3-b2a3-0b507a008811 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:18.773216 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-887ac711-586b-43b3-b2a3-0b507a008811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:18.776728 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 871/1756] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:48:18 2022] DELETE /identity/v3/users/f89e108317d04b06835388e315172037 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:18.782007 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff103de-cd31-411b-8f18-43eea57a5c90 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:18.785890 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff103de-cd31-411b-8f18-43eea57a5c90 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:18.785890 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff103de-cd31-411b-8f18-43eea57a5c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:18.895544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-bff103de-cd31-411b-8f18-43eea57a5c90 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:18.907989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:18.909152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-bff103de-cd31-411b-8f18-43eea57a5c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:18.997963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ac711-586b-43b3-b2a3-0b507a008811 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:19.020926 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:19.152163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 872/1757] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:19.155946 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45488fbc-276d-4391-bf7b-71eb6dce9269 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:19.155946 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45488fbc-276d-4391-bf7b-71eb6dce9269 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:19.156111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45488fbc-276d-4391-bf7b-71eb6dce9269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:19.220061 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-45488fbc-276d-4391-bf7b-71eb6dce9269 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:19.222046 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-45488fbc-276d-4391-bf7b-71eb6dce9269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:19.223636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe194d6-e979-4461-80f5-f9f67ec6b3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:19.224843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe194d6-e979-4461-80f5-f9f67ec6b3c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:19.225672 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe194d6-e979-4461-80f5-f9f67ec6b3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:19.225961 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe194d6-e979-4461-80f5-f9f67ec6b3c2 admin admin] PATH_INFO: `/v3/projects/da8889f26f1143fe937c5a46045794e1/users/fd46f457256d4098b86f40d0fa82f3a1/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:19.234249 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 873/1758] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:19.235949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800708a4-00d1-49df-941b-f9f69cee8dc9 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:19.236304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800708a4-00d1-49df-941b-f9f69cee8dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:19.236999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800708a4-00d1-49df-941b-f9f69cee8dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:19.271704 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-800708a4-00d1-49df-941b-f9f69cee8dc9 None None] MFA Rules not processed for user `942b54c39ae5403c8ae3bcca1c31bf1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:19.275198 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-800708a4-00d1-49df-941b-f9f69cee8dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:19.279659 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe194d6-e979-4461-80f5-f9f67ec6b3c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8889f26f1143fe937c5a46045794e1, user_id=fd46f457256d4098b86f40d0fa82f3a1, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:19.298250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe194d6-e979-4461-80f5-f9f67ec6b3c2 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:19.417246 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 874/1759] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Aug 24 16:48:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:19.424319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec9e5e-6aea-4045-bb5e-1dc4894811f3 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:19.424319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec9e5e-6aea-4045-bb5e-1dc4894811f3 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:19.424319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec9e5e-6aea-4045-bb5e-1dc4894811f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:19.442330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-84ec9e5e-6aea-4045-bb5e-1dc4894811f3 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:19.447261 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-84ec9e5e-6aea-4045-bb5e-1dc4894811f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:19.460384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 886/1760] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:18 2022] PUT /identity/v3/projects/da8889f26f1143fe937c5a46045794e1/users/fd46f457256d4098b86f40d0fa82f3a1/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:19.463939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe194d6-e979-4461-80f5-f9f67ec6b3c2 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:19.469794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-1fe194d6-e979-4461-80f5-f9f67ec6b3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:19.583783 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 875/1761] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:19.588960 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec9e5e-6aea-4045-bb5e-1dc4894811f3 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:19.593508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-84ec9e5e-6aea-4045-bb5e-1dc4894811f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:19.710531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe194d6-e979-4461-80f5-f9f67ec6b3c2 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:19.710788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec9e5e-6aea-4045-bb5e-1dc4894811f3 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:19.714535 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a5ee8-246f-4ca1-8e0f-3deb769a5430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:19.715213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a5ee8-246f-4ca1-8e0f-3deb769a5430 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:19.715498 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a5ee8-246f-4ca1-8e0f-3deb769a5430 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:19.715779 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a5ee8-246f-4ca1-8e0f-3deb769a5430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:19.729161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dae0d6-6a58-4b35-8e65-2bb60fe39dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:19.731452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dae0d6-6a58-4b35-8e65-2bb60fe39dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:19.733276 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dae0d6-6a58-4b35-8e65-2bb60fe39dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:19.733762 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dae0d6-6a58-4b35-8e65-2bb60fe39dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:19.766703 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a5ee8-246f-4ca1-8e0f-3deb769a5430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:19.779423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a5ee8-246f-4ca1-8e0f-3deb769a5430 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:19.785411 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dae0d6-6a58-4b35-8e65-2bb60fe39dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:19.786873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dae0d6-6a58-4b35-8e65-2bb60fe39dc8 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:19.810656 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 887/1762] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:19.812822 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4696c2-9e6b-4fb9-b1bb-ab1528a8dcc2 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:19.813649 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4696c2-9e6b-4fb9-b1bb-ab1528a8dcc2 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:19.814874 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4696c2-9e6b-4fb9-b1bb-ab1528a8dcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:19.815264 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 876/1763] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:19.816349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dae0d6-6a58-4b35-8e65-2bb60fe39dc8 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:19.831491 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dae0d6-6a58-4b35-8e65-2bb60fe39dc8 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:19.837590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cf1d2-eb93-44dd-a787-be105ac9663a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:19.853332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cf1d2-eb93-44dd-a787-be105ac9663a admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:19.853829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cf1d2-eb93-44dd-a787-be105ac9663a admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:19.854603 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cf1d2-eb93-44dd-a787-be105ac9663a admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:19.858367 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888cf1d2-eb93-44dd-a787-be105ac9663a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:48:19.858837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888cf1d2-eb93-44dd-a787-be105ac9663a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:19.860845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888cf1d2-eb93-44dd-a787-be105ac9663a admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:19.888615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 877/1764] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:48:19.893491 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3a4696c2-9e6b-4fb9-b1bb-ab1528a8dcc2 None None] MFA Rules not processed for user `fd46f457256d4098b86f40d0fa82f3a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:19.894517 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cf1d2-eb93-44dd-a787-be105ac9663a admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:19.894810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3a4696c2-9e6b-4fb9-b1bb-ab1528a8dcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:19.907875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cf1d2-eb93-44dd-a787-be105ac9663a admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:19.919211 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a751622-dc62-4eda-a144-c78939a782fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:19.921913 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a751622-dc62-4eda-a144-c78939a782fe admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:19.922405 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a751622-dc62-4eda-a144-c78939a782fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:19.922872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a751622-dc62-4eda-a144-c78939a782fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:19.923679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a751622-dc62-4eda-a144-c78939a782fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:19.926414 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a751622-dc62-4eda-a144-c78939a782fe admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:20.026989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 888/1765] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:20.032261 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4696c2-9e6b-4fb9-b1bb-ab1528a8dcc2 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:20.052603 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4696c2-9e6b-4fb9-b1bb-ab1528a8dcc2 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:20.060424 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b936c3-f058-49a4-ac60-b3e91dfcece2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:20.061362 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b936c3-f058-49a4-ac60-b3e91dfcece2 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:20.062253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b936c3-f058-49a4-ac60-b3e91dfcece2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:20.062693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b936c3-f058-49a4-ac60-b3e91dfcece2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:20.063651 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b936c3-f058-49a4-ac60-b3e91dfcece2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:20.066502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b936c3-f058-49a4-ac60-b3e91dfcece2 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:20.101644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 878/1766] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:48:19 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:20.106199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a751622-dc62-4eda-a144-c78939a782fe admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:20.210281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 889/1767] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:48:20 2022] POST /identity/v3/projects => generated 367 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:20.214587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b936c3-f058-49a4-ac60-b3e91dfcece2 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:20.263600 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a751622-dc62-4eda-a144-c78939a782fe admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:20.367381 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b223b-6605-41c4-ae95-063ed50bb3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:20.373778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b936c3-f058-49a4-ac60-b3e91dfcece2 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:20.373925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b223b-6605-41c4-ae95-063ed50bb3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:20.373925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b223b-6605-41c4-ae95-063ed50bb3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:20.373925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b223b-6605-41c4-ae95-063ed50bb3f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:20.373925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b223b-6605-41c4-ae95-063ed50bb3f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:20.376954 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b223b-6605-41c4-ae95-063ed50bb3f5 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:20.386837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69470975-2a1c-4105-85be-1ff5969889a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:20.387730 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69470975-2a1c-4105-85be-1ff5969889a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:20.388181 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69470975-2a1c-4105-85be-1ff5969889a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:20.388684 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69470975-2a1c-4105-85be-1ff5969889a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:20.389546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69470975-2a1c-4105-85be-1ff5969889a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:20.391947 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69470975-2a1c-4105-85be-1ff5969889a3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:20.551819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 890/1768] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:48:20 2022] POST /identity/v3/users => generated 290 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:20.569405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69470975-2a1c-4105-85be-1ff5969889a3 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:20.574166 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 879/1769] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:48:20 2022] POST /identity/v3/users => generated 291 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:20.578167 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b223b-6605-41c4-ae95-063ed50bb3f5 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:20.606966 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69470975-2a1c-4105-85be-1ff5969889a3 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:20.610938 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b223b-6605-41c4-ae95-063ed50bb3f5 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:20.619863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e272a38-944c-4326-b0eb-b36dcc4cf8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:20.620521 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e272a38-944c-4326-b0eb-b36dcc4cf8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:20.620804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e272a38-944c-4326-b0eb-b36dcc4cf8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:20.621080 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e272a38-944c-4326-b0eb-b36dcc4cf8f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:20.623461 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22f95c-4a08-45bf-a1fc-75779cf0ed97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:20.624553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22f95c-4a08-45bf-a1fc-75779cf0ed97 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:20.625156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22f95c-4a08-45bf-a1fc-75779cf0ed97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:20.625632 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22f95c-4a08-45bf-a1fc-75779cf0ed97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:20.626693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22f95c-4a08-45bf-a1fc-75779cf0ed97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:20.628711 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22f95c-4a08-45bf-a1fc-75779cf0ed97 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:20.629929 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e272a38-944c-4326-b0eb-b36dcc4cf8f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:20.631059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e272a38-944c-4326-b0eb-b36dcc4cf8f7 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:20.648711 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 880/1770] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:20.657442 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22f95c-4a08-45bf-a1fc-75779cf0ed97 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:20.657575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 891/1771] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:20.661307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e272a38-944c-4326-b0eb-b36dcc4cf8f7 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:20.667836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22f95c-4a08-45bf-a1fc-75779cf0ed97 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:20.675177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a7dd4-4e40-4143-92e7-ae008ce20858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:20.677154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a7dd4-4e40-4143-92e7-ae008ce20858 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:20.678273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a7dd4-4e40-4143-92e7-ae008ce20858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:20.678917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a7dd4-4e40-4143-92e7-ae008ce20858 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:20.685290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a7dd4-4e40-4143-92e7-ae008ce20858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:20.685290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a7dd4-4e40-4143-92e7-ae008ce20858 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:20.702310 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e272a38-944c-4326-b0eb-b36dcc4cf8f7 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:20.705563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 881/1772] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:20.706789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a7dd4-4e40-4143-92e7-ae008ce20858 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:20.714743 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8095c-1e40-4c9c-9eda-b2d7f32544e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:20.716032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8095c-1e40-4c9c-9eda-b2d7f32544e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:20.719366 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8095c-1e40-4c9c-9eda-b2d7f32544e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:20.719888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8095c-1e40-4c9c-9eda-b2d7f32544e7 admin admin] PATH_INFO: `/v3/projects/55844ac4c72f4334b6e02c7d0b96cfb0/users/423717395bf84e3d92789abddca17b2c/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:20.721552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a7dd4-4e40-4143-92e7-ae008ce20858 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:20.729833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806fe1a6-8439-416c-8c67-49c847f08e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:20.731328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fe1a6-8439-416c-8c67-49c847f08e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:20.731677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fe1a6-8439-416c-8c67-49c847f08e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:20.732033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fe1a6-8439-416c-8c67-49c847f08e0c admin admin] PATH_INFO: `/v3/projects/380233635980454b8d8a6e4501b28d34/users/0870365dd6f64e32b9461ef7df450669/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:20.777771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806fe1a6-8439-416c-8c67-49c847f08e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=380233635980454b8d8a6e4501b28d34, user_id=0870365dd6f64e32b9461ef7df450669, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:20.786244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806fe1a6-8439-416c-8c67-49c847f08e0c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:20.799268 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:20.812074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8095c-1e40-4c9c-9eda-b2d7f32544e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55844ac4c72f4334b6e02c7d0b96cfb0, user_id=423717395bf84e3d92789abddca17b2c, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:20.819685 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8095c-1e40-4c9c-9eda-b2d7f32544e7 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:20.909165 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 892/1773] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:20 2022] PUT /identity/v3/projects/55844ac4c72f4334b6e02c7d0b96cfb0/users/423717395bf84e3d92789abddca17b2c/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:20.912796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8095c-1e40-4c9c-9eda-b2d7f32544e7 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:20.916289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 882/1774] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:20 2022] PUT /identity/v3/projects/380233635980454b8d8a6e4501b28d34/users/0870365dd6f64e32b9461ef7df450669/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:20.930483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2f961-43cd-4598-b392-b0c100199073 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:20.934711 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2f961-43cd-4598-b392-b0c100199073 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:20.934711 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2f961-43cd-4598-b392-b0c100199073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:21.045313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8095c-1e40-4c9c-9eda-b2d7f32544e7 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:21.053193 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82cee1-6ffb-4648-bb99-084378744b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:21.054284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82cee1-6ffb-4648-bb99-084378744b57 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:21.054575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82cee1-6ffb-4648-bb99-084378744b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:21.054912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82cee1-6ffb-4648-bb99-084378744b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:21.055575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82cee1-6ffb-4648-bb99-084378744b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:21.067021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82cee1-6ffb-4648-bb99-084378744b57 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:21.067164 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-d9a2f961-43cd-4598-b392-b0c100199073 None None] MFA Rules not processed for user `0870365dd6f64e32b9461ef7df450669`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:21.067164 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d9a2f961-43cd-4598-b392-b0c100199073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:21.120527 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 893/1775] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:48:21.127750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82cee1-6ffb-4648-bb99-084378744b57 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:21.150641 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:21.164342 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82cee1-6ffb-4648-bb99-084378744b57 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:21.173665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006ea85-4177-49b3-9241-9a2b162c8a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:21.174707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006ea85-4177-49b3-9241-9a2b162c8a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:21.174960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006ea85-4177-49b3-9241-9a2b162c8a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:21.175236 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006ea85-4177-49b3-9241-9a2b162c8a19 admin admin] PATH_INFO: `/v3/projects/55844ac4c72f4334b6e02c7d0b96cfb0/users/423717395bf84e3d92789abddca17b2c/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:21.179963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006ea85-4177-49b3-9241-9a2b162c8a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55844ac4c72f4334b6e02c7d0b96cfb0, user_id=423717395bf84e3d92789abddca17b2c, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:21.186444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006ea85-4177-49b3-9241-9a2b162c8a19 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:21.215155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 894/1776] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:21 2022] PUT /identity/v3/projects/55844ac4c72f4334b6e02c7d0b96cfb0/users/423717395bf84e3d92789abddca17b2c/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:21.218445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2dbcb-fda9-4579-9b61-d456838779c2 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:21.219144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2dbcb-fda9-4579-9b61-d456838779c2 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:21.219614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2dbcb-fda9-4579-9b61-d456838779c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:21.277796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9dd2dbcb-fda9-4579-9b61-d456838779c2 None None] MFA Rules not processed for user `423717395bf84e3d92789abddca17b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:21.280865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9dd2dbcb-fda9-4579-9b61-d456838779c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:21.380445 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 883/1777] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:21.386190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2f961-43cd-4598-b392-b0c100199073 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:21.401952 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2f961-43cd-4598-b392-b0c100199073 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:21.495464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35351c-d469-4282-8970-3fdba9dc18d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:21.496085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35351c-d469-4282-8970-3fdba9dc18d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:21.496365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35351c-d469-4282-8970-3fdba9dc18d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:21.496681 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35351c-d469-4282-8970-3fdba9dc18d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:21.497450 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35351c-d469-4282-8970-3fdba9dc18d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:21.499903 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35351c-d469-4282-8970-3fdba9dc18d2 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:21.581883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 895/1778] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:21.599244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2dbcb-fda9-4579-9b61-d456838779c2 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:21.599244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9dd2dbcb-fda9-4579-9b61-d456838779c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:21.710468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 884/1779] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:48:21 2022] POST /identity/v3/projects => generated 381 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:21.713516 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35351c-d469-4282-8970-3fdba9dc18d2 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:21.837240 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2dbcb-fda9-4579-9b61-d456838779c2 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:21.889234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35351c-d469-4282-8970-3fdba9dc18d2 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:21.894605 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843ce85-dbb4-4a0c-9258-caec36c62c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:21.895446 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843ce85-dbb4-4a0c-9258-caec36c62c43 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:21.895882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843ce85-dbb4-4a0c-9258-caec36c62c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:21.896134 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843ce85-dbb4-4a0c-9258-caec36c62c43 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:21.897292 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843ce85-dbb4-4a0c-9258-caec36c62c43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:21.915162 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843ce85-dbb4-4a0c-9258-caec36c62c43 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:22.015636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77de09-326d-4fef-843a-cd1ba5830ff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:22.016897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77de09-326d-4fef-843a-cd1ba5830ff7 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:22.017281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77de09-326d-4fef-843a-cd1ba5830ff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:22.017713 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77de09-326d-4fef-843a-cd1ba5830ff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:22.080034 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 885/1780] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:48:21 2022] POST /identity/v3/users => generated 297 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:22.082788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843ce85-dbb4-4a0c-9258-caec36c62c43 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:22.122878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843ce85-dbb4-4a0c-9258-caec36c62c43 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:22.143513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65eb77-03a5-411a-ae8f-91320e2264ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:22.143513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65eb77-03a5-411a-ae8f-91320e2264ca admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:22.143513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65eb77-03a5-411a-ae8f-91320e2264ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:22.143513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65eb77-03a5-411a-ae8f-91320e2264ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:22.143864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65eb77-03a5-411a-ae8f-91320e2264ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:22.145592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65eb77-03a5-411a-ae8f-91320e2264ca admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:22.188304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 886/1781] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:22.189418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65eb77-03a5-411a-ae8f-91320e2264ca admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:22.214153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65eb77-03a5-411a-ae8f-91320e2264ca admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:22.241711 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd248f-1b3a-4b7c-8d59-816c11137725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:22.242411 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd248f-1b3a-4b7c-8d59-816c11137725 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:22.242699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd248f-1b3a-4b7c-8d59-816c11137725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:22.244763 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd248f-1b3a-4b7c-8d59-816c11137725 admin admin] PATH_INFO: `/v3/projects/8db6c7a70db4430b8c5e2596394d92d6/users/ab1c9b7842ab4ea9a13985ae055e969c/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:22.252259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77de09-326d-4fef-843a-cd1ba5830ff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:22.255956 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77de09-326d-4fef-843a-cd1ba5830ff7 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:22.270317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 896/1782] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:22.273143 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc32228-0ca6-4dce-83f4-f806d86b656f None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:22.275719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc32228-0ca6-4dce-83f4-f806d86b656f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:22.275719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc32228-0ca6-4dce-83f4-f806d86b656f None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:22.275719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 897/1783] 10.222.0.70 () {54 vars in 921 bytes} [Wed Aug 24 16:48:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:48:22.333273 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bed3f-ac79-4e79-834d-1a75b4adbd34 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:22.334206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bed3f-ac79-4e79-834d-1a75b4adbd34 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:22.334322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bed3f-ac79-4e79-834d-1a75b4adbd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:22.338538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd248f-1b3a-4b7c-8d59-816c11137725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db6c7a70db4430b8c5e2596394d92d6, user_id=ab1c9b7842ab4ea9a13985ae055e969c, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:22.350218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd248f-1b3a-4b7c-8d59-816c11137725 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:22.393390 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b83bed3f-ac79-4e79-834d-1a75b4adbd34 None None] MFA Rules not processed for user `e333fca7790244f6a4ebceabb55faba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:22.402155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b83bed3f-ac79-4e79-834d-1a75b4adbd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:22.491096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 887/1784] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:22 2022] PUT /identity/v3/projects/8db6c7a70db4430b8c5e2596394d92d6/users/ab1c9b7842ab4ea9a13985ae055e969c/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:22.501429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd248f-1b3a-4b7c-8d59-816c11137725 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:22.636337 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 898/1785] 10.222.0.70 () {58 vars in 1018 bytes} [Wed Aug 24 16:48:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:48:22.639413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bed3f-ac79-4e79-834d-1a75b4adbd34 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:22.644963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b83bed3f-ac79-4e79-834d-1a75b4adbd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:22.682362 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bed3f-ac79-4e79-834d-1a75b4adbd34 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:22.701536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3eef2b-ab98-404f-befb-117f0651d3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:22.704380 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3eef2b-ab98-404f-befb-117f0651d3ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:22.704380 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3eef2b-ab98-404f-befb-117f0651d3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:22.704380 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3eef2b-ab98-404f-befb-117f0651d3ba admin admin] PATH_INFO: `/v3/projects/67baf73b311f4f39a880a8651d690758` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:22.708301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3eef2b-ab98-404f-befb-117f0651d3ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=67baf73b311f4f39a880a8651d690758)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:22.710614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3eef2b-ab98-404f-befb-117f0651d3ba admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:22.724333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd248f-1b3a-4b7c-8d59-816c11137725 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:22.744721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939488c7-cb4b-414b-aea7-c73568887b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:22.747718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939488c7-cb4b-414b-aea7-c73568887b20 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:22.749647 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939488c7-cb4b-414b-aea7-c73568887b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:22.750551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939488c7-cb4b-414b-aea7-c73568887b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:22.751619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939488c7-cb4b-414b-aea7-c73568887b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:22.759024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939488c7-cb4b-414b-aea7-c73568887b20 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:22.800986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 888/1786] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:22.804575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939488c7-cb4b-414b-aea7-c73568887b20 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:22.810232 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-939488c7-cb4b-414b-aea7-c73568887b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:22.841789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:22.901976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4e3eef2b-ab98-404f-befb-117f0651d3ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67baf73b311f4f39a880a8651d690758', 'request_id': 'req-4e3eef2b-ab98-404f-befb-117f0651d3ba'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:22.913641 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4e3eef2b-ab98-404f-befb-117f0651d3ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67baf73b311f4f39a880a8651d690758', 'request_id': 'req-4e3eef2b-ab98-404f-befb-117f0651d3ba'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:22.929816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4e3eef2b-ab98-404f-befb-117f0651d3ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67baf73b311f4f39a880a8651d690758', 'request_id': 'req-4e3eef2b-ab98-404f-befb-117f0651d3ba'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:22.939690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939488c7-cb4b-414b-aea7-c73568887b20 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:23.012364 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4e3eef2b-ab98-404f-befb-117f0651d3ba admin admin] The token cache is being invalidate because project 67baf73b311f4f39a880a8651d690758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:48:23.012694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4e3eef2b-ab98-404f-befb-117f0651d3ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:23.016831 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4e3eef2b-ab98-404f-befb-117f0651d3ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:23.019039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 899/1787] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:48:22 2022] DELETE /identity/v3/projects/67baf73b311f4f39a880a8651d690758 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:23.021445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3eef2b-ab98-404f-befb-117f0651d3ba admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:23.021639 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4e3eef2b-ab98-404f-befb-117f0651d3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:23.069993 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab35b0b-7baa-4f23-af9f-82b6d3f58cad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:23.078834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab35b0b-7baa-4f23-af9f-82b6d3f58cad service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:23.081366 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab35b0b-7baa-4f23-af9f-82b6d3f58cad service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:23.081812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab35b0b-7baa-4f23-af9f-82b6d3f58cad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:23.090472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-aab35b0b-7baa-4f23-af9f-82b6d3f58cad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:23.143297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:23.179322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3eef2b-ab98-404f-befb-117f0651d3ba admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:23.186761 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7b6a4-11f7-4828-8257-de7b28dd8dcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:23.187560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7b6a4-11f7-4828-8257-de7b28dd8dcf service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:23.187914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7b6a4-11f7-4828-8257-de7b28dd8dcf service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:23.188289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7b6a4-11f7-4828-8257-de7b28dd8dcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:23.190241 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-07e7b6a4-11f7-4828-8257-de7b28dd8dcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:23.226166 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7b6a4-11f7-4828-8257-de7b28dd8dcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:23.232747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7b6a4-11f7-4828-8257-de7b28dd8dcf service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:23.287920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 900/1788] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:23.289110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7b6a4-11f7-4828-8257-de7b28dd8dcf service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:23.293497 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-07e7b6a4-11f7-4828-8257-de7b28dd8dcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:23.302030 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab35b0b-7baa-4f23-af9f-82b6d3f58cad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:23.304524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab35b0b-7baa-4f23-af9f-82b6d3f58cad service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:23.341680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 889/1789] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:23.341680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46ab74-1904-4088-8715-93f17dd26a03 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:23.341680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46ab74-1904-4088-8715-93f17dd26a03 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:23.341680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46ab74-1904-4088-8715-93f17dd26a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:23.428656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-7d46ab74-1904-4088-8715-93f17dd26a03 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:23.431784 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7d46ab74-1904-4088-8715-93f17dd26a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:23.467726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 890/1790] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:23.475502 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfbd8b-d5ec-4228-b1dc-c6cda995ac32 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:23.475796 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfbd8b-d5ec-4228-b1dc-c6cda995ac32 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:23.476077 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfbd8b-d5ec-4228-b1dc-c6cda995ac32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:23.483659 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7b6a4-11f7-4828-8257-de7b28dd8dcf service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:23.497930 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9795d9-ebec-426c-8cd0-5b4a4d6aa28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:23.500554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9795d9-ebec-426c-8cd0-5b4a4d6aa28c admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:23.500922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9795d9-ebec-426c-8cd0-5b4a4d6aa28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:23.501208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9795d9-ebec-426c-8cd0-5b4a4d6aa28c admin admin] PATH_INFO: `/v3/projects/8db6c7a70db4430b8c5e2596394d92d6/users/ab1c9b7842ab4ea9a13985ae055e969c/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:23.506232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9795d9-ebec-426c-8cd0-5b4a4d6aa28c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db6c7a70db4430b8c5e2596394d92d6, user_id=ab1c9b7842ab4ea9a13985ae055e969c, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:23.515083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9795d9-ebec-426c-8cd0-5b4a4d6aa28c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:23.548914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 901/1791] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:23 2022] PUT /identity/v3/projects/8db6c7a70db4430b8c5e2596394d92d6/users/ab1c9b7842ab4ea9a13985ae055e969c/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:23.560433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9934f-2e39-43c2-8f79-6ab7219c28c3 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:23.561216 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9934f-2e39-43c2-8f79-6ab7219c28c3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:23.561705 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9934f-2e39-43c2-8f79-6ab7219c28c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:23.565142 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-e8dfbd8b-d5ec-4228-b1dc-c6cda995ac32 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:23.566663 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e8dfbd8b-d5ec-4228-b1dc-c6cda995ac32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:23.626501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-10a9934f-2e39-43c2-8f79-6ab7219c28c3 None None] MFA Rules not processed for user `ab1c9b7842ab4ea9a13985ae055e969c`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:23.627920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-10a9934f-2e39-43c2-8f79-6ab7219c28c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:23.831615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 902/1792] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:48:23.855111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 891/1793] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:23.859213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9934f-2e39-43c2-8f79-6ab7219c28c3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:23.895245 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9934f-2e39-43c2-8f79-6ab7219c28c3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:23.919723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6dcb0-ef66-4f16-b2c7-c9700ff36ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:23.920360 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6dcb0-ef66-4f16-b2c7-c9700ff36ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:23.920672 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6dcb0-ef66-4f16-b2c7-c9700ff36ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:23.928731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6dcb0-ef66-4f16-b2c7-c9700ff36ce4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:23.931260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6dcb0-ef66-4f16-b2c7-c9700ff36ce4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:48:23.932732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6dcb0-ef66-4f16-b2c7-c9700ff36ce4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:23.934401 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6dcb0-ef66-4f16-b2c7-c9700ff36ce4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:24.001122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfbd8b-d5ec-4228-b1dc-c6cda995ac32 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:24.010047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 903/1794] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:48:24.025205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfbd8b-d5ec-4228-b1dc-c6cda995ac32 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:24.039173 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:24.073512 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e1cfe-9d1b-4090-8c32-4d456969d647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:24.077832 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e1cfe-9d1b-4090-8c32-4d456969d647 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:24.077832 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e1cfe-9d1b-4090-8c32-4d456969d647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:24.077832 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e1cfe-9d1b-4090-8c32-4d456969d647 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:24.100732 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e1cfe-9d1b-4090-8c32-4d456969d647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:24.108834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e1cfe-9d1b-4090-8c32-4d456969d647 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:24.295021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 892/1795] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:48:23 2022] POST /identity/v3/projects => generated 373 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:48:24.300079 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6dcb0-ef66-4f16-b2c7-c9700ff36ce4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:24.313187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6dcb0-ef66-4f16-b2c7-c9700ff36ce4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:24.438204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fb227-b435-4a1f-92d8-8026b61091ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:24.438901 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fb227-b435-4a1f-92d8-8026b61091ab admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:24.439299 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fb227-b435-4a1f-92d8-8026b61091ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:24.439505 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fb227-b435-4a1f-92d8-8026b61091ab admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:24.440463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fb227-b435-4a1f-92d8-8026b61091ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:24.442550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fb227-b435-4a1f-92d8-8026b61091ab admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:24.571579 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 904/1796] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:48:24 2022] POST /identity/v3/users => generated 287 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:24.575379 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e1cfe-9d1b-4090-8c32-4d456969d647 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:24.595071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e1cfe-9d1b-4090-8c32-4d456969d647 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:24.603837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fb227-b435-4a1f-92d8-8026b61091ab admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:24.606016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-063fb227-b435-4a1f-92d8-8026b61091ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:24.617555 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c188f-2a76-4821-a371-e53ccbfdca59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:24.625157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c188f-2a76-4821-a371-e53ccbfdca59 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:24.625157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c188f-2a76-4821-a371-e53ccbfdca59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:24.625157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c188f-2a76-4821-a371-e53ccbfdca59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:24.625157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c188f-2a76-4821-a371-e53ccbfdca59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:24.625157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c188f-2a76-4821-a371-e53ccbfdca59 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:24.637685 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 893/1797] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:48:24.642042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c188f-2a76-4821-a371-e53ccbfdca59 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:24.676505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c188f-2a76-4821-a371-e53ccbfdca59 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:24.677240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb9a92-e1b5-4167-8092-bb7148ed84de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:24.678314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb9a92-e1b5-4167-8092-bb7148ed84de admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:24.679005 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb9a92-e1b5-4167-8092-bb7148ed84de admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:24.679466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb9a92-e1b5-4167-8092-bb7148ed84de admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:24.685543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fb227-b435-4a1f-92d8-8026b61091ab admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:24.686702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb9a92-e1b5-4167-8092-bb7148ed84de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:24.689178 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb9a92-e1b5-4167-8092-bb7148ed84de admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:24.689745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58722e9-5dec-4dd6-93ba-fd7f27d31dc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:24.691172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58722e9-5dec-4dd6-93ba-fd7f27d31dc7 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:24.691471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58722e9-5dec-4dd6-93ba-fd7f27d31dc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:24.691750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58722e9-5dec-4dd6-93ba-fd7f27d31dc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:24.716985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 894/1798] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:48:24.720117 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb9a92-e1b5-4167-8092-bb7148ed84de admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:24.745000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb9a92-e1b5-4167-8092-bb7148ed84de admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:24.759674 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b5968-3296-4bc4-869f-3c5843137b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:24.759674 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b5968-3296-4bc4-869f-3c5843137b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:24.759674 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b5968-3296-4bc4-869f-3c5843137b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:24.759674 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b5968-3296-4bc4-869f-3c5843137b7c admin admin] PATH_INFO: `/v3/projects/c82aef4f789e49b8930cc0410a5dc80c/users/0706c9a5c3294a898a4b452c872449ed/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:24.795409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b5968-3296-4bc4-869f-3c5843137b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c82aef4f789e49b8930cc0410a5dc80c, user_id=0706c9a5c3294a898a4b452c872449ed, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:24.795409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b5968-3296-4bc4-869f-3c5843137b7c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:24.821524 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58722e9-5dec-4dd6-93ba-fd7f27d31dc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:24.824389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58722e9-5dec-4dd6-93ba-fd7f27d31dc7 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:24.833487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:24.847189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 905/1799] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:48:24.893079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 895/1800] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:24 2022] PUT /identity/v3/projects/c82aef4f789e49b8930cc0410a5dc80c/users/0706c9a5c3294a898a4b452c872449ed/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:24.904360 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a481cf0-48b8-4c24-b1c2-48d459705dd0 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:24.904917 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a481cf0-48b8-4c24-b1c2-48d459705dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:24.905347 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a481cf0-48b8-4c24-b1c2-48d459705dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:24.944317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5a481cf0-48b8-4c24-b1c2-48d459705dd0 None None] MFA Rules not processed for user `0706c9a5c3294a898a4b452c872449ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:24.948712 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5a481cf0-48b8-4c24-b1c2-48d459705dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:25.067162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 906/1801] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:25.080843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b5968-3296-4bc4-869f-3c5843137b7c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:25.221167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a481cf0-48b8-4c24-b1c2-48d459705dd0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:25.224696 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5a481cf0-48b8-4c24-b1c2-48d459705dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:25.227614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b5968-3296-4bc4-869f-3c5843137b7c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:25.244680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b46e1f-8482-4271-841e-dc69ac0de893 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:25.247841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b46e1f-8482-4271-841e-dc69ac0de893 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:25.248249 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b46e1f-8482-4271-841e-dc69ac0de893 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:25.248724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b46e1f-8482-4271-841e-dc69ac0de893 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:25.275569 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a481cf0-48b8-4c24-b1c2-48d459705dd0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:25.285963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b46e1f-8482-4271-841e-dc69ac0de893 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:25.294243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a522f-e89c-40fe-9130-13a5f09941a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:25.297030 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a522f-e89c-40fe-9130-13a5f09941a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:25.298039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a522f-e89c-40fe-9130-13a5f09941a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:25.298329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a522f-e89c-40fe-9130-13a5f09941a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:25.300660 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b46e1f-8482-4271-841e-dc69ac0de893 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:25.304112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a19a522f-e89c-40fe-9130-13a5f09941a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:25.325204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:25.337379 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 896/1802] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:25.445879 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a522f-e89c-40fe-9130-13a5f09941a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:25.449439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a522f-e89c-40fe-9130-13a5f09941a9 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:25.464822 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 907/1803] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:48:26.005484 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e9664-7fac-4de3-bcf7-6334198c04b2 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:26.007331 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e9664-7fac-4de3-bcf7-6334198c04b2 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:26.008337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e9664-7fac-4de3-bcf7-6334198c04b2 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:26.011763 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 897/1804] 10.222.0.70 () {54 vars in 921 bytes} [Wed Aug 24 16:48:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:48:26.047977 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38eeec-bbd2-46cd-973d-0250fba5fc0a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:26.048574 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38eeec-bbd2-46cd-973d-0250fba5fc0a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:26.048789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38eeec-bbd2-46cd-973d-0250fba5fc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:26.086037 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ca38eeec-bbd2-46cd-973d-0250fba5fc0a None None] MFA Rules not processed for user `e333fca7790244f6a4ebceabb55faba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:26.089951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ca38eeec-bbd2-46cd-973d-0250fba5fc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:26.314664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 908/1805] 10.222.0.70 () {58 vars in 1018 bytes} [Wed Aug 24 16:48:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:26.450649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e9664-7fac-4de3-bcf7-6334198c04b2 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:26.499060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e9664-7fac-4de3-bcf7-6334198c04b2 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:26.536004 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c8fd1-fba8-49f0-bb2e-5500789a8b90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:26.536966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c8fd1-fba8-49f0-bb2e-5500789a8b90 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:26.537391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c8fd1-fba8-49f0-bb2e-5500789a8b90 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:26.537833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c8fd1-fba8-49f0-bb2e-5500789a8b90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:26.560350 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c8fd1-fba8-49f0-bb2e-5500789a8b90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:26.560350 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c8fd1-fba8-49f0-bb2e-5500789a8b90 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:26.591560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 898/1806] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:48:27.498753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:27.759817 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:28.998294 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:29.978024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:30.363854 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:30.769646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:30.828924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:31.292174 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:31.449762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:31.621357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:31.850144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:36.699032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38eeec-bbd2-46cd-973d-0250fba5fc0a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:36.705224 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ca38eeec-bbd2-46cd-973d-0250fba5fc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:36.772881 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38eeec-bbd2-46cd-973d-0250fba5fc0a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:36.777663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fcfc6-f06e-4eb0-b055-420bb7382f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:36.779570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fcfc6-f06e-4eb0-b055-420bb7382f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:36.779570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fcfc6-f06e-4eb0-b055-420bb7382f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:36.779570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fcfc6-f06e-4eb0-b055-420bb7382f4d admin admin] PATH_INFO: `/v3/users/fd46f457256d4098b86f40d0fa82f3a1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:36.780137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fcfc6-f06e-4eb0-b055-420bb7382f4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd46f457256d4098b86f40d0fa82f3a1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:36.782135 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fcfc6-f06e-4eb0-b055-420bb7382f4d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:36.962059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0a2fcfc6-f06e-4eb0-b055-420bb7382f4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd46f457256d4098b86f40d0fa82f3a1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:36.962635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0a2fcfc6-f06e-4eb0-b055-420bb7382f4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd46f457256d4098b86f40d0fa82f3a1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:36.963232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0a2fcfc6-f06e-4eb0-b055-420bb7382f4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd46f457256d4098b86f40d0fa82f3a1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:37.039976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0a2fcfc6-f06e-4eb0-b055-420bb7382f4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd46f457256d4098b86f40d0fa82f3a1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:37.047264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0a2fcfc6-f06e-4eb0-b055-420bb7382f4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd46f457256d4098b86f40d0fa82f3a1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:37.064102 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 909/1807] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:48:36 2022] DELETE /identity/v3/users/fd46f457256d4098b86f40d0fa82f3a1 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:37.068686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78682e65-c734-47b4-b79c-a2322b6cfad7 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:37.069183 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78682e65-c734-47b4-b79c-a2322b6cfad7 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:37.069278 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78682e65-c734-47b4-b79c-a2322b6cfad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:37.089175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-78682e65-c734-47b4-b79c-a2322b6cfad7 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:37.090387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-78682e65-c734-47b4-b79c-a2322b6cfad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:37.155312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 899/1808] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:37.164882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fcfc6-f06e-4eb0-b055-420bb7382f4d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:37.166323 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0a2fcfc6-f06e-4eb0-b055-420bb7382f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:37.243493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fcfc6-f06e-4eb0-b055-420bb7382f4d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:37.248390 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a646787-f16d-4817-b331-522ab20e054f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:37.249048 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a646787-f16d-4817-b331-522ab20e054f service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:37.249326 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a646787-f16d-4817-b331-522ab20e054f service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:37.249601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a646787-f16d-4817-b331-522ab20e054f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:37.259556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a646787-f16d-4817-b331-522ab20e054f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:37.264358 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a646787-f16d-4817-b331-522ab20e054f service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:37.272935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 910/1809] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:37.668168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78682e65-c734-47b4-b79c-a2322b6cfad7 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:37.669685 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-78682e65-c734-47b4-b79c-a2322b6cfad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:37.682959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78682e65-c734-47b4-b79c-a2322b6cfad7 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:37.687688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13385a94-2464-45b7-a543-dda0008c681c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:37.688507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13385a94-2464-45b7-a543-dda0008c681c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:37.689010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13385a94-2464-45b7-a543-dda0008c681c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:37.689283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13385a94-2464-45b7-a543-dda0008c681c admin admin] PATH_INFO: `/v3/projects/da8889f26f1143fe937c5a46045794e1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:37.690586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13385a94-2464-45b7-a543-dda0008c681c admin admin] RBAC: Authorizing `identity:delete_project(project_id=da8889f26f1143fe937c5a46045794e1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:37.700031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13385a94-2464-45b7-a543-dda0008c681c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:37.829263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-13385a94-2464-45b7-a543-dda0008c681c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da8889f26f1143fe937c5a46045794e1', 'request_id': 'req-13385a94-2464-45b7-a543-dda0008c681c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:37.837494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-13385a94-2464-45b7-a543-dda0008c681c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da8889f26f1143fe937c5a46045794e1', 'request_id': 'req-13385a94-2464-45b7-a543-dda0008c681c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:37.845012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-13385a94-2464-45b7-a543-dda0008c681c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da8889f26f1143fe937c5a46045794e1', 'request_id': 'req-13385a94-2464-45b7-a543-dda0008c681c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:37.918682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-13385a94-2464-45b7-a543-dda0008c681c admin admin] The token cache is being invalidate because project da8889f26f1143fe937c5a46045794e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:48:37.919077 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-13385a94-2464-45b7-a543-dda0008c681c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:37.919702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-13385a94-2464-45b7-a543-dda0008c681c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:37.921308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 900/1810] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:48:37 2022] DELETE /identity/v3/projects/da8889f26f1143fe937c5a46045794e1 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:37.925394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a646787-f16d-4817-b331-522ab20e054f service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:37.935309 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a646787-f16d-4817-b331-522ab20e054f service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:37.943538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:37.994480 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04daba7-c5e2-4f5e-9cd7-6603d6d19946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:37.998759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04daba7-c5e2-4f5e-9cd7-6603d6d19946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:37.998759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04daba7-c5e2-4f5e-9cd7-6603d6d19946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:37.998759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04daba7-c5e2-4f5e-9cd7-6603d6d19946 admin admin] PATH_INFO: `/v3/users/423717395bf84e3d92789abddca17b2c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:37.998759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04daba7-c5e2-4f5e-9cd7-6603d6d19946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=423717395bf84e3d92789abddca17b2c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:37.998759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04daba7-c5e2-4f5e-9cd7-6603d6d19946 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:38.167724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a04daba7-c5e2-4f5e-9cd7-6603d6d19946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '423717395bf84e3d92789abddca17b2c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:38.168492 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a04daba7-c5e2-4f5e-9cd7-6603d6d19946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '423717395bf84e3d92789abddca17b2c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:38.169096 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a04daba7-c5e2-4f5e-9cd7-6603d6d19946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '423717395bf84e3d92789abddca17b2c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:38.260048 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a04daba7-c5e2-4f5e-9cd7-6603d6d19946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '423717395bf84e3d92789abddca17b2c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:38.268010 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a04daba7-c5e2-4f5e-9cd7-6603d6d19946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '423717395bf84e3d92789abddca17b2c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:38.284940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 911/1811] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:48:37 2022] DELETE /identity/v3/users/423717395bf84e3d92789abddca17b2c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:38.710136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13385a94-2464-45b7-a543-dda0008c681c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:38.713470 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-13385a94-2464-45b7-a543-dda0008c681c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:38.727670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13385a94-2464-45b7-a543-dda0008c681c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:38.775815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:38.790241 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07962780-df21-4e98-a83f-6a0526f337c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:38.790871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07962780-df21-4e98-a83f-6a0526f337c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:38.791151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07962780-df21-4e98-a83f-6a0526f337c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:38.791930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07962780-df21-4e98-a83f-6a0526f337c9 admin admin] PATH_INFO: `/v3/projects/55844ac4c72f4334b6e02c7d0b96cfb0` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:38.793059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07962780-df21-4e98-a83f-6a0526f337c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55844ac4c72f4334b6e02c7d0b96cfb0)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:38.794980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07962780-df21-4e98-a83f-6a0526f337c9 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:38.898320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-07962780-df21-4e98-a83f-6a0526f337c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55844ac4c72f4334b6e02c7d0b96cfb0', 'request_id': 'req-07962780-df21-4e98-a83f-6a0526f337c9'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:38.905383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-07962780-df21-4e98-a83f-6a0526f337c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55844ac4c72f4334b6e02c7d0b96cfb0', 'request_id': 'req-07962780-df21-4e98-a83f-6a0526f337c9'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:38.911314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-07962780-df21-4e98-a83f-6a0526f337c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55844ac4c72f4334b6e02c7d0b96cfb0', 'request_id': 'req-07962780-df21-4e98-a83f-6a0526f337c9'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:38.977326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-07962780-df21-4e98-a83f-6a0526f337c9 admin admin] The token cache is being invalidate because project 55844ac4c72f4334b6e02c7d0b96cfb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:48:38.977550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-07962780-df21-4e98-a83f-6a0526f337c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:38.978171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-07962780-df21-4e98-a83f-6a0526f337c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:38.980360 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 901/1812] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:48:38 2022] DELETE /identity/v3/projects/55844ac4c72f4334b6e02c7d0b96cfb0 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:38.985564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50253103-f0eb-4ba3-bf85-81653222d20f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:38.986023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50253103-f0eb-4ba3-bf85-81653222d20f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:38.986136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50253103-f0eb-4ba3-bf85-81653222d20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:39.006095 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-50253103-f0eb-4ba3-bf85-81653222d20f None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:39.007408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-50253103-f0eb-4ba3-bf85-81653222d20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:39.066341 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 912/1813] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:39.074206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdce86-aafd-4820-97a0-a4273068a3dc None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:39.074498 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdce86-aafd-4820-97a0-a4273068a3dc None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:39.074789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdce86-aafd-4820-97a0-a4273068a3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:39.096724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-effdce86-aafd-4820-97a0-a4273068a3dc None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:39.098024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-effdce86-aafd-4820-97a0-a4273068a3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:39.106828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 902/1814] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:39.110924 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50253103-f0eb-4ba3-bf85-81653222d20f None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:39.120760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50253103-f0eb-4ba3-bf85-81653222d20f None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:39.125363 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af005d-3dc2-43f9-a736-f9bdd8d8814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:39.126038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af005d-3dc2-43f9-a736-f9bdd8d8814d admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:39.126314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af005d-3dc2-43f9-a736-f9bdd8d8814d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:39.126587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af005d-3dc2-43f9-a736-f9bdd8d8814d admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:39.127231 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af005d-3dc2-43f9-a736-f9bdd8d8814d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:48:39.127538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af005d-3dc2-43f9-a736-f9bdd8d8814d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:39.129148 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af005d-3dc2-43f9-a736-f9bdd8d8814d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:39.141692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 913/1815] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:48:39.145908 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effdce86-aafd-4820-97a0-a4273068a3dc None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:39.155688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effdce86-aafd-4820-97a0-a4273068a3dc None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:39.160493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad3314-a0d7-4279-b478-ce12e1daed32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:39.167068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad3314-a0d7-4279-b478-ce12e1daed32 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:39.167068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad3314-a0d7-4279-b478-ce12e1daed32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:39.167068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad3314-a0d7-4279-b478-ce12e1daed32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:39.167068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad3314-a0d7-4279-b478-ce12e1daed32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:39.167068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad3314-a0d7-4279-b478-ce12e1daed32 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:39.260962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 903/1816] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:48:39 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:39.265104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af005d-3dc2-43f9-a736-f9bdd8d8814d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:39.271942 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af005d-3dc2-43f9-a736-f9bdd8d8814d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:39.285264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:39.345377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2cb00-237a-4949-a18c-992bb8781aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:39.346009 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2cb00-237a-4949-a18c-992bb8781aca admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:39.346626 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2cb00-237a-4949-a18c-992bb8781aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:39.347097 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2cb00-237a-4949-a18c-992bb8781aca admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:39.347729 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2cb00-237a-4949-a18c-992bb8781aca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:39.349713 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2cb00-237a-4949-a18c-992bb8781aca admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:39.430967 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 914/1817] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:48:39 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:39.435794 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad3314-a0d7-4279-b478-ce12e1daed32 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:39.445982 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad3314-a0d7-4279-b478-ce12e1daed32 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:39.450615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c0550-bacf-428e-b411-51c19239aa58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:39.451224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c0550-bacf-428e-b411-51c19239aa58 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:39.451507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c0550-bacf-428e-b411-51c19239aa58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:39.451781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c0550-bacf-428e-b411-51c19239aa58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:39.452445 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c0550-bacf-428e-b411-51c19239aa58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:39.454045 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c0550-bacf-428e-b411-51c19239aa58 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:39.464096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 904/1818] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:48:39.467973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2cb00-237a-4949-a18c-992bb8781aca admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:39.478068 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2cb00-237a-4949-a18c-992bb8781aca admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:39.483266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae85e9-c594-47f4-91f4-5acacee26403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:39.483805 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae85e9-c594-47f4-91f4-5acacee26403 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:39.484087 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae85e9-c594-47f4-91f4-5acacee26403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:39.484362 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae85e9-c594-47f4-91f4-5acacee26403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:39.485037 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae85e9-c594-47f4-91f4-5acacee26403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:39.486890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae85e9-c594-47f4-91f4-5acacee26403 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:39.498830 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 915/1819] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:48:39.502773 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c0550-bacf-428e-b411-51c19239aa58 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:39.512505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c0550-bacf-428e-b411-51c19239aa58 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:39.517245 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c0ab5-82a4-4db3-a88e-2e08f84d9ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:39.517899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c0ab5-82a4-4db3-a88e-2e08f84d9ef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:39.518243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c0ab5-82a4-4db3-a88e-2e08f84d9ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:39.518524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c0ab5-82a4-4db3-a88e-2e08f84d9ef4 admin admin] PATH_INFO: `/v3/projects/7f142c6098f94f9187e24c107750c589/users/817214b217ba426c8a190f729ae862d1/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:39.533092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c0ab5-82a4-4db3-a88e-2e08f84d9ef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f142c6098f94f9187e24c107750c589, user_id=817214b217ba426c8a190f729ae862d1, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:39.535914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c0ab5-82a4-4db3-a88e-2e08f84d9ef4 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:39.610497 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 905/1820] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:39 2022] PUT /identity/v3/projects/7f142c6098f94f9187e24c107750c589/users/817214b217ba426c8a190f729ae862d1/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:39.611765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b23bee-7545-4dae-9b1a-ce4bf5643d67 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:39.612104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b23bee-7545-4dae-9b1a-ce4bf5643d67 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:39.612384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b23bee-7545-4dae-9b1a-ce4bf5643d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:39.643664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-d4b23bee-7545-4dae-9b1a-ce4bf5643d67 None None] MFA Rules not processed for user `817214b217ba426c8a190f729ae862d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:39.644926 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d4b23bee-7545-4dae-9b1a-ce4bf5643d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:39.714887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 916/1821] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:48:39.719464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c0ab5-82a4-4db3-a88e-2e08f84d9ef4 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:39.787869 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c0ab5-82a4-4db3-a88e-2e08f84d9ef4 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:39.793269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780795bd-ef5a-4012-a187-7101479e9952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:39.793960 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780795bd-ef5a-4012-a187-7101479e9952 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:39.794254 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780795bd-ef5a-4012-a187-7101479e9952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:39.794531 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780795bd-ef5a-4012-a187-7101479e9952 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:39.795166 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780795bd-ef5a-4012-a187-7101479e9952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:39.797120 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780795bd-ef5a-4012-a187-7101479e9952 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:39.816566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:39.888209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 906/1822] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:48:39 2022] POST /identity/v3/projects => generated 375 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:39.895202 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b23bee-7545-4dae-9b1a-ce4bf5643d67 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:39.908307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b23bee-7545-4dae-9b1a-ce4bf5643d67 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:39.966711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793ab08-bad5-40df-b529-67947769ae31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:39.967394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793ab08-bad5-40df-b529-67947769ae31 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:39.967678 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793ab08-bad5-40df-b529-67947769ae31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:39.967952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793ab08-bad5-40df-b529-67947769ae31 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:39.968632 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793ab08-bad5-40df-b529-67947769ae31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:39.970549 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793ab08-bad5-40df-b529-67947769ae31 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:40.053397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 917/1823] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:48:39 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:48:40.054696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780795bd-ef5a-4012-a187-7101479e9952 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:40.067163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780795bd-ef5a-4012-a187-7101479e9952 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:40.073694 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee336fb4-b5a1-4b86-8162-ae934a4acaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:40.080163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee336fb4-b5a1-4b86-8162-ae934a4acaec admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:40.080163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee336fb4-b5a1-4b86-8162-ae934a4acaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:40.080163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee336fb4-b5a1-4b86-8162-ae934a4acaec admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:40.080163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee336fb4-b5a1-4b86-8162-ae934a4acaec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:40.080163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee336fb4-b5a1-4b86-8162-ae934a4acaec admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:40.093704 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 907/1824] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:48:40.095104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793ab08-bad5-40df-b529-67947769ae31 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:40.106383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793ab08-bad5-40df-b529-67947769ae31 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:40.111851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acdf37-ccb6-42a3-9de7-07f5c08811ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:40.112576 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acdf37-ccb6-42a3-9de7-07f5c08811ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:40.112858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acdf37-ccb6-42a3-9de7-07f5c08811ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:40.113135 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acdf37-ccb6-42a3-9de7-07f5c08811ab admin admin] PATH_INFO: `/v3/projects/f2793935938d4eee8f072437aae81881/users/b20f1f1beb334e8ca39fd2804e8b0867/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:40.127605 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:40.132756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12acdf37-ccb6-42a3-9de7-07f5c08811ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2793935938d4eee8f072437aae81881, user_id=b20f1f1beb334e8ca39fd2804e8b0867, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:40.136364 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12acdf37-ccb6-42a3-9de7-07f5c08811ab admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:40.226248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 918/1825] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:40 2022] PUT /identity/v3/projects/f2793935938d4eee8f072437aae81881/users/b20f1f1beb334e8ca39fd2804e8b0867/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:40.227308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee336fb4-b5a1-4b86-8162-ae934a4acaec admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:40.239476 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee336fb4-b5a1-4b86-8162-ae934a4acaec admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:40.315416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a9ec6-205a-423e-9bc7-43ce37e37836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:40.322796 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a9ec6-205a-423e-9bc7-43ce37e37836 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:40.323094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a9ec6-205a-423e-9bc7-43ce37e37836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:40.323376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a9ec6-205a-423e-9bc7-43ce37e37836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:40.324060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a9ec6-205a-423e-9bc7-43ce37e37836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:40.325716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a9ec6-205a-423e-9bc7-43ce37e37836 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:40.356443 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 908/1826] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:48:40.360662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acdf37-ccb6-42a3-9de7-07f5c08811ab admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:40.374616 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acdf37-ccb6-42a3-9de7-07f5c08811ab admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:40.384236 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ada66-682a-473a-8890-00817530af59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:40.385284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ada66-682a-473a-8890-00817530af59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:40.385562 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ada66-682a-473a-8890-00817530af59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:40.385900 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ada66-682a-473a-8890-00817530af59 admin admin] PATH_INFO: `/v3/projects/f2793935938d4eee8f072437aae81881/users/b20f1f1beb334e8ca39fd2804e8b0867/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:40.387917 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ada66-682a-473a-8890-00817530af59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2793935938d4eee8f072437aae81881, user_id=b20f1f1beb334e8ca39fd2804e8b0867, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:40.390570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ada66-682a-473a-8890-00817530af59 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:40.402090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 919/1827] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:48:40 2022] PUT /identity/v3/projects/f2793935938d4eee8f072437aae81881/users/b20f1f1beb334e8ca39fd2804e8b0867/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:48:40.413139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82addcc-f25a-4d49-bfff-4f0056a1597b None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:40.413436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82addcc-f25a-4d49-bfff-4f0056a1597b None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:40.413718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82addcc-f25a-4d49-bfff-4f0056a1597b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:40.442255 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-a82addcc-f25a-4d49-bfff-4f0056a1597b None None] MFA Rules not processed for user `b20f1f1beb334e8ca39fd2804e8b0867`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:40.443488 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a82addcc-f25a-4d49-bfff-4f0056a1597b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:40.526271 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 909/1828] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:40.548731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ada66-682a-473a-8890-00817530af59 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:40.550461 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e81ada66-682a-473a-8890-00817530af59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:40.657804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ada66-682a-473a-8890-00817530af59 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:40.661600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76e5d6-2a50-4c59-a6a9-8a2248baad5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:40.662326 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76e5d6-2a50-4c59-a6a9-8a2248baad5b service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:40.662607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76e5d6-2a50-4c59-a6a9-8a2248baad5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:40.662893 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76e5d6-2a50-4c59-a6a9-8a2248baad5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:40.741858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76e5d6-2a50-4c59-a6a9-8a2248baad5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:40.743558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76e5d6-2a50-4c59-a6a9-8a2248baad5b service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:40.757835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 920/1829] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:50.518258 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=70 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:50.751855 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:57.791657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82addcc-f25a-4d49-bfff-4f0056a1597b None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:57.878328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82addcc-f25a-4d49-bfff-4f0056a1597b None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:57.884765 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193fea7b-b920-4756-9fd6-87e28d65d97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:57.885983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193fea7b-b920-4756-9fd6-87e28d65d97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:57.886268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193fea7b-b920-4756-9fd6-87e28d65d97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:57.886722 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193fea7b-b920-4756-9fd6-87e28d65d97b admin admin] PATH_INFO: `/v3/users/817214b217ba426c8a190f729ae862d1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:57.888238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193fea7b-b920-4756-9fd6-87e28d65d97b admin admin] RBAC: Authorizing `identity:delete_user(user_id=817214b217ba426c8a190f729ae862d1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:57.890454 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193fea7b-b920-4756-9fd6-87e28d65d97b admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:58.098608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-193fea7b-b920-4756-9fd6-87e28d65d97b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '817214b217ba426c8a190f729ae862d1'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:58.099255 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-193fea7b-b920-4756-9fd6-87e28d65d97b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '817214b217ba426c8a190f729ae862d1'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:58.099873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-193fea7b-b920-4756-9fd6-87e28d65d97b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '817214b217ba426c8a190f729ae862d1'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:58.106731 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-193fea7b-b920-4756-9fd6-87e28d65d97b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '817214b217ba426c8a190f729ae862d1'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:58.202441 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-193fea7b-b920-4756-9fd6-87e28d65d97b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '817214b217ba426c8a190f729ae862d1'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:58.221629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 910/1830] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:48:57 2022] DELETE /identity/v3/users/817214b217ba426c8a190f729ae862d1 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:58.226243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649479a3-6d0a-4d83-968d-0deca21bbee6 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:58.226700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649479a3-6d0a-4d83-968d-0deca21bbee6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:58.226829 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649479a3-6d0a-4d83-968d-0deca21bbee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:58.247545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-649479a3-6d0a-4d83-968d-0deca21bbee6 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:58.248769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-649479a3-6d0a-4d83-968d-0deca21bbee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:58.310547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 921/1831] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:48:58.321015 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193fea7b-b920-4756-9fd6-87e28d65d97b admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:58.322458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-193fea7b-b920-4756-9fd6-87e28d65d97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:58.379561 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193fea7b-b920-4756-9fd6-87e28d65d97b admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:58.383645 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf3739-3427-4ca3-a785-dc2aff706f3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:58.384255 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf3739-3427-4ca3-a785-dc2aff706f3a service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:58.384577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf3739-3427-4ca3-a785-dc2aff706f3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:58.384841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf3739-3427-4ca3-a785-dc2aff706f3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:58.395215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf3739-3427-4ca3-a785-dc2aff706f3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:58.397134 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf3739-3427-4ca3-a785-dc2aff706f3a service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:58.407621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 911/1832] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:58.975836 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649479a3-6d0a-4d83-968d-0deca21bbee6 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:58.978106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-649479a3-6d0a-4d83-968d-0deca21bbee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:59.007285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649479a3-6d0a-4d83-968d-0deca21bbee6 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:59.017067 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:59.017738 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:59.018022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:59.018301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942 admin admin] PATH_INFO: `/v3/projects/7f142c6098f94f9187e24c107750c589` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:59.020076 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f142c6098f94f9187e24c107750c589)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:59.022296 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:59.169485 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f142c6098f94f9187e24c107750c589', 'request_id': 'req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:59.177023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f142c6098f94f9187e24c107750c589', 'request_id': 'req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:59.184261 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f142c6098f94f9187e24c107750c589', 'request_id': 'req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:59.269578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942 admin admin] The token cache is being invalidate because project 7f142c6098f94f9187e24c107750c589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:48:59.270162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:59.270700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:59.272373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 922/1833] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:48:58 2022] DELETE /identity/v3/projects/7f142c6098f94f9187e24c107750c589 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:59.276547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf3739-3427-4ca3-a785-dc2aff706f3a service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:59.287080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf3739-3427-4ca3-a785-dc2aff706f3a service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:59.309508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:59.350459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f844087-c304-4a5d-ab53-9c543b7245cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:59.351091 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f844087-c304-4a5d-ab53-9c543b7245cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:59.351372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f844087-c304-4a5d-ab53-9c543b7245cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:59.351670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f844087-c304-4a5d-ab53-9c543b7245cf admin admin] PATH_INFO: `/v3/users/b20f1f1beb334e8ca39fd2804e8b0867` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:59.352838 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f844087-c304-4a5d-ab53-9c543b7245cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b20f1f1beb334e8ca39fd2804e8b0867)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:59.354861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f844087-c304-4a5d-ab53-9c543b7245cf admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:59.578642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4f844087-c304-4a5d-ab53-9c543b7245cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b20f1f1beb334e8ca39fd2804e8b0867'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:59.579624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4f844087-c304-4a5d-ab53-9c543b7245cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b20f1f1beb334e8ca39fd2804e8b0867'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:59.580332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4f844087-c304-4a5d-ab53-9c543b7245cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b20f1f1beb334e8ca39fd2804e8b0867'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:59.582849 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:59.584464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:59.588276 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4f844087-c304-4a5d-ab53-9c543b7245cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b20f1f1beb334e8ca39fd2804e8b0867'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:59.597352 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8f3ea-eb72-4c40-83e6-4f6899a9d942 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:59.602186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfda75f-7456-4ec4-b1f9-874f454416a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:59.602812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfda75f-7456-4ec4-b1f9-874f454416a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:59.603090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfda75f-7456-4ec4-b1f9-874f454416a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:59.603363 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfda75f-7456-4ec4-b1f9-874f454416a7 admin admin] PATH_INFO: `/v3/users/0870365dd6f64e32b9461ef7df450669` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:59.604509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfda75f-7456-4ec4-b1f9-874f454416a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0870365dd6f64e32b9461ef7df450669)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:59.606959 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfda75f-7456-4ec4-b1f9-874f454416a7 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:59.690667 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4f844087-c304-4a5d-ab53-9c543b7245cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b20f1f1beb334e8ca39fd2804e8b0867'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:59.708948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 912/1834] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:48:59 2022] DELETE /identity/v3/users/b20f1f1beb334e8ca39fd2804e8b0867 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:59.842149 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9bfda75f-7456-4ec4-b1f9-874f454416a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0870365dd6f64e32b9461ef7df450669'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:59.842938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9bfda75f-7456-4ec4-b1f9-874f454416a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0870365dd6f64e32b9461ef7df450669'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:59.843693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9bfda75f-7456-4ec4-b1f9-874f454416a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0870365dd6f64e32b9461ef7df450669'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:59.902841 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9bfda75f-7456-4ec4-b1f9-874f454416a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0870365dd6f64e32b9461ef7df450669'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:59.911851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9bfda75f-7456-4ec4-b1f9-874f454416a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0870365dd6f64e32b9461ef7df450669'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:59.925534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:59.936672 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 923/1835] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:48:59 2022] DELETE /identity/v3/users/0870365dd6f64e32b9461ef7df450669 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:48:59.939068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a620e-f0b0-4a49-8c4c-5b124f194cc0 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:59.939652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a620e-f0b0-4a49-8c4c-5b124f194cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:59.940000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a620e-f0b0-4a49-8c4c-5b124f194cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:59.986895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-0d7a620e-f0b0-4a49-8c4c-5b124f194cc0 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:59.988560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-0d7a620e-f0b0-4a49-8c4c-5b124f194cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:00.114537 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 913/1836] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:00.123528 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfda75f-7456-4ec4-b1f9-874f454416a7 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:00.125556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9bfda75f-7456-4ec4-b1f9-874f454416a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:00.134962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068ec7a-daec-4c85-a69c-02cf430e0c5c None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:00.135245 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068ec7a-daec-4c85-a69c-02cf430e0c5c None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:00.135534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068ec7a-daec-4c85-a69c-02cf430e0c5c None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:00.137721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 914/1837] 10.222.0.70 () {54 vars in 921 bytes} [Wed Aug 24 16:49:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:49:00.149335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d47fe-ac72-4f3a-8599-5d16fb74bcf1 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:00.150493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d47fe-ac72-4f3a-8599-5d16fb74bcf1 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:00.151018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d47fe-ac72-4f3a-8599-5d16fb74bcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:00.195740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-ef2d47fe-ac72-4f3a-8599-5d16fb74bcf1 None None] MFA Rules not processed for user `e333fca7790244f6a4ebceabb55faba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:00.199700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ef2d47fe-ac72-4f3a-8599-5d16fb74bcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:00.252460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfda75f-7456-4ec4-b1f9-874f454416a7 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:00.256738 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fdb41-8bc5-4fc1-90bd-10d4a1a60585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:00.257376 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fdb41-8bc5-4fc1-90bd-10d4a1a60585 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:00.257657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fdb41-8bc5-4fc1-90bd-10d4a1a60585 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:00.258063 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fdb41-8bc5-4fc1-90bd-10d4a1a60585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:00.281919 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fdb41-8bc5-4fc1-90bd-10d4a1a60585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:00.283377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fdb41-8bc5-4fc1-90bd-10d4a1a60585 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:00.308780 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 924/1838] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:00.312043 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fdb41-8bc5-4fc1-90bd-10d4a1a60585 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:00.316501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2d1fdb41-8bc5-4fc1-90bd-10d4a1a60585 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:00.325002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:00.336897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fdb41-8bc5-4fc1-90bd-10d4a1a60585 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:00.345120 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:00.346255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:00.346770 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:00.347028 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b admin admin] PATH_INFO: `/v3/projects/f2793935938d4eee8f072437aae81881` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:00.348569 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2793935938d4eee8f072437aae81881)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:00.351958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:00.411415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 915/1839] 10.222.0.70 () {58 vars in 1018 bytes} [Wed Aug 24 16:49:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:00.495052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2793935938d4eee8f072437aae81881', 'request_id': 'req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:00.503327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2793935938d4eee8f072437aae81881', 'request_id': 'req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:00.509652 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2793935938d4eee8f072437aae81881', 'request_id': 'req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:00.594554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b admin admin] The token cache is being invalidate because project f2793935938d4eee8f072437aae81881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:00.595460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:00.596455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:00.601612 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 925/1840] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:00 2022] DELETE /identity/v3/projects/f2793935938d4eee8f072437aae81881 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:00.604270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c6aba-ff8a-4d1f-b50d-a354437ba716 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:00.609383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c6aba-ff8a-4d1f-b50d-a354437ba716 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:00.609383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c6aba-ff8a-4d1f-b50d-a354437ba716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:00.660987 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:00.664727 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-de6c6aba-ff8a-4d1f-b50d-a354437ba716 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:00.665840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:00.668743 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-de6c6aba-ff8a-4d1f-b50d-a354437ba716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:00.776769 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 916/1841] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:00.778369 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cd55c-faf7-477c-8584-30828d99241c None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:00.778659 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cd55c-faf7-477c-8584-30828d99241c None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:00.778937 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cd55c-faf7-477c-8584-30828d99241c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:00.779636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0f9d1-5aff-4d77-a6cd-820ed717ba1b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:00.786061 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fcedf-06e1-4924-9508-4028fe64bf50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:00.786728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fcedf-06e1-4924-9508-4028fe64bf50 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:00.788372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fcedf-06e1-4924-9508-4028fe64bf50 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:00.788372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fcedf-06e1-4924-9508-4028fe64bf50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:00.790938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3e5fcedf-06e1-4924-9508-4028fe64bf50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:00.798541 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-7c4cd55c-faf7-477c-8584-30828d99241c None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:00.799723 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7c4cd55c-faf7-477c-8584-30828d99241c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:00.804975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fcedf-06e1-4924-9508-4028fe64bf50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:00.811148 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fcedf-06e1-4924-9508-4028fe64bf50 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:00.811284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 917/1842] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:00.813156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cd55c-faf7-477c-8584-30828d99241c None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:00.818656 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 926/1843] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:00.823211 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cd55c-faf7-477c-8584-30828d99241c None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:00.833561 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29996532-d72e-473e-a460-ab6a463a0db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:00.839158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29996532-d72e-473e-a460-ab6a463a0db4 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:00.839158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29996532-d72e-473e-a460-ab6a463a0db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:00.839158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29996532-d72e-473e-a460-ab6a463a0db4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:00.839158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29996532-d72e-473e-a460-ab6a463a0db4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:49:00.839158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29996532-d72e-473e-a460-ab6a463a0db4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:00.839158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29996532-d72e-473e-a460-ab6a463a0db4 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:00.852636 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 918/1844] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:49:00.856739 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fcedf-06e1-4924-9508-4028fe64bf50 service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:00.870301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29996532-d72e-473e-a460-ab6a463a0db4 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:00.873419 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-29996532-d72e-473e-a460-ab6a463a0db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:00.875479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fcedf-06e1-4924-9508-4028fe64bf50 service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:00.887440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3e7ef-2f11-4f8f-968a-aab7d1b6ee0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:00.888188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3e7ef-2f11-4f8f-968a-aab7d1b6ee0d admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:00.888619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3e7ef-2f11-4f8f-968a-aab7d1b6ee0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:00.888809 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3e7ef-2f11-4f8f-968a-aab7d1b6ee0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:00.889517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3e7ef-2f11-4f8f-968a-aab7d1b6ee0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:00.891442 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3e7ef-2f11-4f8f-968a-aab7d1b6ee0d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:00.893275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29996532-d72e-473e-a460-ab6a463a0db4 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:00.896369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:00.897808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:00.898515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:00.898878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:00.899201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37 admin admin] PATH_INFO: `/v3/projects/380233635980454b8d8a6e4501b28d34` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:00.900464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=380233635980454b8d8a6e4501b28d34)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:00.902454 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:01.005897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 927/1845] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:00 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:01.009976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3e7ef-2f11-4f8f-968a-aab7d1b6ee0d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:01.040317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '380233635980454b8d8a6e4501b28d34', 'request_id': 'req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:01.047577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '380233635980454b8d8a6e4501b28d34', 'request_id': 'req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:01.053060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '380233635980454b8d8a6e4501b28d34', 'request_id': 'req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:01.071718 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3e7ef-2f11-4f8f-968a-aab7d1b6ee0d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:01.128787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224ea02-7d99-4d5d-b951-27ec29e94641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:01.129333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224ea02-7d99-4d5d-b951-27ec29e94641 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:01.129618 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224ea02-7d99-4d5d-b951-27ec29e94641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:01.129895 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224ea02-7d99-4d5d-b951-27ec29e94641 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:01.130587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224ea02-7d99-4d5d-b951-27ec29e94641 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:01.132480 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224ea02-7d99-4d5d-b951-27ec29e94641 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:01.141267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37 admin admin] The token cache is being invalidate because project 380233635980454b8d8a6e4501b28d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:01.141563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:01.145403 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:01.145403 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 919/1846] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:00 2022] DELETE /identity/v3/projects/380233635980454b8d8a6e4501b28d34 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:01.147842 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:01.149226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:01.161962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64e78f-98a7-49b5-89b2-ae8a7d218a37 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:01.166669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceacd06-a955-4d1e-9218-cb2476c41e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:01.167280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceacd06-a955-4d1e-9218-cb2476c41e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:01.168093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceacd06-a955-4d1e-9218-cb2476c41e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:01.168373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceacd06-a955-4d1e-9218-cb2476c41e28 admin admin] PATH_INFO: `/v3/users/ab1c9b7842ab4ea9a13985ae055e969c` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:01.169624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceacd06-a955-4d1e-9218-cb2476c41e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab1c9b7842ab4ea9a13985ae055e969c)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:01.171533 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceacd06-a955-4d1e-9218-cb2476c41e28 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:01.265496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 928/1847] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:01 2022] POST /identity/v3/users => generated 288 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:01.269434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224ea02-7d99-4d5d-b951-27ec29e94641 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:01.270993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0224ea02-7d99-4d5d-b951-27ec29e94641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:01.282974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224ea02-7d99-4d5d-b951-27ec29e94641 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:01.287832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51857d4c-c6f0-4ee0-906d-50beeb68cb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:01.288424 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51857d4c-c6f0-4ee0-906d-50beeb68cb70 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:01.288724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51857d4c-c6f0-4ee0-906d-50beeb68cb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:01.288999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51857d4c-c6f0-4ee0-906d-50beeb68cb70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:01.289652 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51857d4c-c6f0-4ee0-906d-50beeb68cb70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:01.291224 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51857d4c-c6f0-4ee0-906d-50beeb68cb70 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:01.302103 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 929/1848] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:49:01.306014 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51857d4c-c6f0-4ee0-906d-50beeb68cb70 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:01.315592 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51857d4c-c6f0-4ee0-906d-50beeb68cb70 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:01.320187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d54dd-40b3-420e-833a-11a0d312bde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:01.320795 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d54dd-40b3-420e-833a-11a0d312bde1 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:01.321080 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d54dd-40b3-420e-833a-11a0d312bde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:01.321356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d54dd-40b3-420e-833a-11a0d312bde1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:01.321980 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d54dd-40b3-420e-833a-11a0d312bde1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:01.323529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d54dd-40b3-420e-833a-11a0d312bde1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:01.333550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 930/1849] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:49:01.337256 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d54dd-40b3-420e-833a-11a0d312bde1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:01.346980 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d54dd-40b3-420e-833a-11a0d312bde1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:01.351571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6531a1-4e03-4039-9040-38d7f1580827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:01.352177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6531a1-4e03-4039-9040-38d7f1580827 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:01.352485 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6531a1-4e03-4039-9040-38d7f1580827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:01.352765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6531a1-4e03-4039-9040-38d7f1580827 admin admin] PATH_INFO: `/v3/projects/7a04faeffed14d078bffa686d5a6aaf7/users/b2d0547b423448e094417e7cca7d67b1/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:01.367030 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6531a1-4e03-4039-9040-38d7f1580827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a04faeffed14d078bffa686d5a6aaf7, user_id=b2d0547b423448e094417e7cca7d67b1, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:01.369940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6531a1-4e03-4039-9040-38d7f1580827 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:01.394342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:01.401147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9ceacd06-a955-4d1e-9218-cb2476c41e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab1c9b7842ab4ea9a13985ae055e969c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:01.401796 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9ceacd06-a955-4d1e-9218-cb2476c41e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab1c9b7842ab4ea9a13985ae055e969c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:01.402926 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9ceacd06-a955-4d1e-9218-cb2476c41e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab1c9b7842ab4ea9a13985ae055e969c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:01.409793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9ceacd06-a955-4d1e-9218-cb2476c41e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab1c9b7842ab4ea9a13985ae055e969c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:01.488956 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 931/1850] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:01 2022] PUT /identity/v3/projects/7a04faeffed14d078bffa686d5a6aaf7/users/b2d0547b423448e094417e7cca7d67b1/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:01.502872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966e20e-85ee-4b12-9a9e-15c453643c92 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:01.503192 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966e20e-85ee-4b12-9a9e-15c453643c92 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:01.503483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966e20e-85ee-4b12-9a9e-15c453643c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:01.507760 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9ceacd06-a955-4d1e-9218-cb2476c41e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab1c9b7842ab4ea9a13985ae055e969c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:01.539204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 920/1851] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:01 2022] DELETE /identity/v3/users/ab1c9b7842ab4ea9a13985ae055e969c => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:01.554240 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-6966e20e-85ee-4b12-9a9e-15c453643c92 None None] MFA Rules not processed for user `b2d0547b423448e094417e7cca7d67b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:01.555464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6966e20e-85ee-4b12-9a9e-15c453643c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:01.670809 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 932/1852] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:01.683364 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceacd06-a955-4d1e-9218-cb2476c41e28 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:01.685281 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-9ceacd06-a955-4d1e-9218-cb2476c41e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:01.792708 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:01.813921 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceacd06-a955-4d1e-9218-cb2476c41e28 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:01.817747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee6ac7-20e8-4587-9780-19ec33b555e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:01.818445 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee6ac7-20e8-4587-9780-19ec33b555e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:01.818903 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee6ac7-20e8-4587-9780-19ec33b555e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:01.819321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee6ac7-20e8-4587-9780-19ec33b555e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:01.830680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee6ac7-20e8-4587-9780-19ec33b555e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:01.832497 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee6ac7-20e8-4587-9780-19ec33b555e2 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:01.866130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 921/1853] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:02.198173 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966e20e-85ee-4b12-9a9e-15c453643c92 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:02.200434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6966e20e-85ee-4b12-9a9e-15c453643c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:02.274039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966e20e-85ee-4b12-9a9e-15c453643c92 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:02.278745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b5110-4d1a-4714-b598-42a00da255c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:02.279382 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b5110-4d1a-4714-b598-42a00da255c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:02.279659 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b5110-4d1a-4714-b598-42a00da255c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:02.279978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b5110-4d1a-4714-b598-42a00da255c1 admin admin] PATH_INFO: `/v3/projects/8db6c7a70db4430b8c5e2596394d92d6` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:02.281102 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b5110-4d1a-4714-b598-42a00da255c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8db6c7a70db4430b8c5e2596394d92d6)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:02.283006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b5110-4d1a-4714-b598-42a00da255c1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:02.425994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-187b5110-4d1a-4714-b598-42a00da255c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8db6c7a70db4430b8c5e2596394d92d6', 'request_id': 'req-187b5110-4d1a-4714-b598-42a00da255c1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:02.432232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-187b5110-4d1a-4714-b598-42a00da255c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8db6c7a70db4430b8c5e2596394d92d6', 'request_id': 'req-187b5110-4d1a-4714-b598-42a00da255c1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:02.438398 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-187b5110-4d1a-4714-b598-42a00da255c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8db6c7a70db4430b8c5e2596394d92d6', 'request_id': 'req-187b5110-4d1a-4714-b598-42a00da255c1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:02.532601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-187b5110-4d1a-4714-b598-42a00da255c1 admin admin] The token cache is being invalidate because project 8db6c7a70db4430b8c5e2596394d92d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:02.532900 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-187b5110-4d1a-4714-b598-42a00da255c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:02.533529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-187b5110-4d1a-4714-b598-42a00da255c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:02.535060 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 933/1854] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:02 2022] DELETE /identity/v3/projects/8db6c7a70db4430b8c5e2596394d92d6 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:02.541968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42305454-47e1-46bd-8ada-06a49b1829ac None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:02.543085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42305454-47e1-46bd-8ada-06a49b1829ac None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:02.543085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42305454-47e1-46bd-8ada-06a49b1829ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:02.569147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-42305454-47e1-46bd-8ada-06a49b1829ac None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:02.570398 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-42305454-47e1-46bd-8ada-06a49b1829ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:02.632379 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 922/1855] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:02.640264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01618c3-9251-4189-a4d3-1c5ffed6d590 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:02.640690 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01618c3-9251-4189-a4d3-1c5ffed6d590 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:02.641127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01618c3-9251-4189-a4d3-1c5ffed6d590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:02.661939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f01618c3-9251-4189-a4d3-1c5ffed6d590 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:02.663227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f01618c3-9251-4189-a4d3-1c5ffed6d590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:02.677266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 934/1856] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:02.681469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42305454-47e1-46bd-8ada-06a49b1829ac None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:02.693115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42305454-47e1-46bd-8ada-06a49b1829ac None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:02.702486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4995238-063a-4727-a572-e8cf3e8c8473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:02.703074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4995238-063a-4727-a572-e8cf3e8c8473 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:02.703354 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4995238-063a-4727-a572-e8cf3e8c8473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:02.703619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4995238-063a-4727-a572-e8cf3e8c8473 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:02.704592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4995238-063a-4727-a572-e8cf3e8c8473 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:49:02.705203 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4995238-063a-4727-a572-e8cf3e8c8473 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:02.707578 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4995238-063a-4727-a572-e8cf3e8c8473 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:02.721857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 923/1857] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:49:02.726728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01618c3-9251-4189-a4d3-1c5ffed6d590 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:02.737852 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01618c3-9251-4189-a4d3-1c5ffed6d590 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:02.742584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b45b67-0f04-4762-8854-e10232ef7c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:02.743461 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b45b67-0f04-4762-8854-e10232ef7c73 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:02.743872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b45b67-0f04-4762-8854-e10232ef7c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:02.744151 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b45b67-0f04-4762-8854-e10232ef7c73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:02.744851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b45b67-0f04-4762-8854-e10232ef7c73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:02.747716 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b45b67-0f04-4762-8854-e10232ef7c73 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:02.875971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 935/1858] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:02 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:02.877197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4995238-063a-4727-a572-e8cf3e8c8473 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:02.887099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4995238-063a-4727-a572-e8cf3e8c8473 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:02.925586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:02.943607 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97fc88-4f71-4a15-99ca-e80b0c207276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:02.944234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97fc88-4f71-4a15-99ca-e80b0c207276 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:02.944560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97fc88-4f71-4a15-99ca-e80b0c207276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:02.944841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97fc88-4f71-4a15-99ca-e80b0c207276 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:02.945473 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97fc88-4f71-4a15-99ca-e80b0c207276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:02.947395 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97fc88-4f71-4a15-99ca-e80b0c207276 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:03.014569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 924/1859] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:02 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:03.019543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b45b67-0f04-4762-8854-e10232ef7c73 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:03.032935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b45b67-0f04-4762-8854-e10232ef7c73 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:03.038490 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5d9a4-01f4-4b2f-b46c-5982d2f1f028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:03.039204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5d9a4-01f4-4b2f-b46c-5982d2f1f028 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:03.039495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5d9a4-01f4-4b2f-b46c-5982d2f1f028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:03.041473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5d9a4-01f4-4b2f-b46c-5982d2f1f028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:03.041473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5d9a4-01f4-4b2f-b46c-5982d2f1f028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:03.042140 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5d9a4-01f4-4b2f-b46c-5982d2f1f028 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:03.053391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 936/1860] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:03.057644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97fc88-4f71-4a15-99ca-e80b0c207276 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:03.069879 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97fc88-4f71-4a15-99ca-e80b0c207276 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:03.075072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c1a75-6a2f-432c-b3b5-3336be5a30b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:03.075685 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c1a75-6a2f-432c-b3b5-3336be5a30b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:03.075969 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c1a75-6a2f-432c-b3b5-3336be5a30b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:03.076244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c1a75-6a2f-432c-b3b5-3336be5a30b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:03.076933 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c1a75-6a2f-432c-b3b5-3336be5a30b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:03.078530 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c1a75-6a2f-432c-b3b5-3336be5a30b1 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:03.090765 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 925/1861] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:03.094327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5d9a4-01f4-4b2f-b46c-5982d2f1f028 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:03.104375 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5d9a4-01f4-4b2f-b46c-5982d2f1f028 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:03.109568 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2353f87-67a3-4e97-94f1-5fe9bf881b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:03.110343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2353f87-67a3-4e97-94f1-5fe9bf881b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:03.110636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2353f87-67a3-4e97-94f1-5fe9bf881b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:03.110921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2353f87-67a3-4e97-94f1-5fe9bf881b19 admin admin] PATH_INFO: `/v3/projects/c3fb217b6f4e43eabb9cb718b77677ca/users/dd903de44365478d99845213d2778406/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:03.127348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:03.142031 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2353f87-67a3-4e97-94f1-5fe9bf881b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3fb217b6f4e43eabb9cb718b77677ca, user_id=dd903de44365478d99845213d2778406, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:03.142031 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2353f87-67a3-4e97-94f1-5fe9bf881b19 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:03.280912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 937/1862] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:03 2022] PUT /identity/v3/projects/c3fb217b6f4e43eabb9cb718b77677ca/users/dd903de44365478d99845213d2778406/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:03.292459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca51b2-faaa-4167-9d94-ae5fdb3bf0be None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:03.292752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca51b2-faaa-4167-9d94-ae5fdb3bf0be None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:03.293030 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca51b2-faaa-4167-9d94-ae5fdb3bf0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:03.319829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-57ca51b2-faaa-4167-9d94-ae5fdb3bf0be None None] MFA Rules not processed for user `dd903de44365478d99845213d2778406`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:03.321090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-57ca51b2-faaa-4167-9d94-ae5fdb3bf0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:03.384981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 926/1863] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:03.390054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2353f87-67a3-4e97-94f1-5fe9bf881b19 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:03.454254 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2353f87-67a3-4e97-94f1-5fe9bf881b19 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:03.458883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27611bc-3af6-458d-988b-785b529d1683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:03.459497 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27611bc-3af6-458d-988b-785b529d1683 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:03.459784 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27611bc-3af6-458d-988b-785b529d1683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:03.460062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27611bc-3af6-458d-988b-785b529d1683 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:03.460729 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27611bc-3af6-458d-988b-785b529d1683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:03.462628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27611bc-3af6-458d-988b-785b529d1683 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:03.552437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 938/1864] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:03 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:03.556487 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca51b2-faaa-4167-9d94-ae5fdb3bf0be None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:03.567829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca51b2-faaa-4167-9d94-ae5fdb3bf0be None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:03.623709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4711e-d888-4586-8f1b-a33402517f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:03.624405 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4711e-d888-4586-8f1b-a33402517f2c admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:03.624715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4711e-d888-4586-8f1b-a33402517f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:03.624988 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4711e-d888-4586-8f1b-a33402517f2c admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:03.625614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4711e-d888-4586-8f1b-a33402517f2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:03.627572 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4711e-d888-4586-8f1b-a33402517f2c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:03.739959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 927/1865] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:03 2022] POST /identity/v3/users => generated 299 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:03.744298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27611bc-3af6-458d-988b-785b529d1683 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:03.755621 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27611bc-3af6-458d-988b-785b529d1683 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:03.760460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3f807-2b23-4aae-9b54-5238efa3b72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:03.761073 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3f807-2b23-4aae-9b54-5238efa3b72c admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:03.761932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3f807-2b23-4aae-9b54-5238efa3b72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:03.762210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3f807-2b23-4aae-9b54-5238efa3b72c admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:03.762862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3f807-2b23-4aae-9b54-5238efa3b72c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:03.764551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3f807-2b23-4aae-9b54-5238efa3b72c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:03.775636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 939/1866] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:03.782496 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4711e-d888-4586-8f1b-a33402517f2c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:03.792717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4711e-d888-4586-8f1b-a33402517f2c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:03.797997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf70a4f-4d46-43e7-8ea6-fd44645059ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:03.798655 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70a4f-4d46-43e7-8ea6-fd44645059ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:03.805278 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70a4f-4d46-43e7-8ea6-fd44645059ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:03.805278 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70a4f-4d46-43e7-8ea6-fd44645059ea admin admin] PATH_INFO: `/v3/projects/c2860711c8994a09b051d8806184fc12/users/d94bccf167fa4d36903818e691d83775/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:03.814751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf70a4f-4d46-43e7-8ea6-fd44645059ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2860711c8994a09b051d8806184fc12, user_id=d94bccf167fa4d36903818e691d83775, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:03.817539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf70a4f-4d46-43e7-8ea6-fd44645059ea admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:03.934521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 928/1867] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:03 2022] PUT /identity/v3/projects/c2860711c8994a09b051d8806184fc12/users/d94bccf167fa4d36903818e691d83775/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:03.938383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3f807-2b23-4aae-9b54-5238efa3b72c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:03.950205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3f807-2b23-4aae-9b54-5238efa3b72c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:04.037263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d176ca-0581-476b-ad1e-d6ba6a3d938d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:04.037263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d176ca-0581-476b-ad1e-d6ba6a3d938d admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:04.037263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d176ca-0581-476b-ad1e-d6ba6a3d938d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:04.037263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d176ca-0581-476b-ad1e-d6ba6a3d938d admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:04.037937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d176ca-0581-476b-ad1e-d6ba6a3d938d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:04.038280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d176ca-0581-476b-ad1e-d6ba6a3d938d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:04.060714 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 940/1868] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:04.061909 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf70a4f-4d46-43e7-8ea6-fd44645059ea admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:04.075549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf70a4f-4d46-43e7-8ea6-fd44645059ea admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:04.083447 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd0442-5270-48d7-80a9-4c18c614a5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:04.084137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd0442-5270-48d7-80a9-4c18c614a5bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:04.084450 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd0442-5270-48d7-80a9-4c18c614a5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:04.084739 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd0442-5270-48d7-80a9-4c18c614a5bf admin admin] PATH_INFO: `/v3/projects/c2860711c8994a09b051d8806184fc12/users/d94bccf167fa4d36903818e691d83775/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:04.087118 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd0442-5270-48d7-80a9-4c18c614a5bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2860711c8994a09b051d8806184fc12, user_id=d94bccf167fa4d36903818e691d83775, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:04.090743 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd0442-5270-48d7-80a9-4c18c614a5bf admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:04.101383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:04.116180 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 929/1869] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:04 2022] PUT /identity/v3/projects/c2860711c8994a09b051d8806184fc12/users/d94bccf167fa4d36903818e691d83775/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:04.120210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc152a-20f9-43ae-bab4-c147cb829d6d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:04.120915 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc152a-20f9-43ae-bab4-c147cb829d6d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:04.121336 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc152a-20f9-43ae-bab4-c147cb829d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:04.156943 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-50fc152a-20f9-43ae-bab4-c147cb829d6d None None] MFA Rules not processed for user `d94bccf167fa4d36903818e691d83775`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:04.158310 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-50fc152a-20f9-43ae-bab4-c147cb829d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:04.211025 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:04.233458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 941/1870] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:04 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:04.246844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd0442-5270-48d7-80a9-4c18c614a5bf admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:04.248380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a7dd0442-5270-48d7-80a9-4c18c614a5bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:04.329264 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd0442-5270-48d7-80a9-4c18c614a5bf admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:04.333116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591eb20-941d-475e-8de1-36bc48d68d7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:04.334027 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591eb20-941d-475e-8de1-36bc48d68d7f service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:04.334438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591eb20-941d-475e-8de1-36bc48d68d7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:04.334839 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591eb20-941d-475e-8de1-36bc48d68d7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:04.400684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591eb20-941d-475e-8de1-36bc48d68d7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:04.402174 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591eb20-941d-475e-8de1-36bc48d68d7f service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:04.411188 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 930/1871] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:14.361930 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=6 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:14.407030 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=48 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:15.125027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc152a-20f9-43ae-bab4-c147cb829d6d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:15.126830 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-50fc152a-20f9-43ae-bab4-c147cb829d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:15.227776 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc152a-20f9-43ae-bab4-c147cb829d6d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:15.228534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b93d4-1555-4fbf-b463-bf4e83e5c600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:15.229205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b93d4-1555-4fbf-b463-bf4e83e5c600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:15.229488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b93d4-1555-4fbf-b463-bf4e83e5c600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:15.229764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b93d4-1555-4fbf-b463-bf4e83e5c600 admin admin] PATH_INFO: `/v3/users/b2d0547b423448e094417e7cca7d67b1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:15.230923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b93d4-1555-4fbf-b463-bf4e83e5c600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2d0547b423448e094417e7cca7d67b1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:15.232922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b93d4-1555-4fbf-b463-bf4e83e5c600 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:15.449389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5b7b93d4-1555-4fbf-b463-bf4e83e5c600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2d0547b423448e094417e7cca7d67b1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:15.450165 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5b7b93d4-1555-4fbf-b463-bf4e83e5c600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2d0547b423448e094417e7cca7d67b1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:15.451286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5b7b93d4-1555-4fbf-b463-bf4e83e5c600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2d0547b423448e094417e7cca7d67b1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:15.527143 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5b7b93d4-1555-4fbf-b463-bf4e83e5c600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2d0547b423448e094417e7cca7d67b1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:15.550073 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5b7b93d4-1555-4fbf-b463-bf4e83e5c600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2d0547b423448e094417e7cca7d67b1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:15.576342 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 942/1872] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:15 2022] DELETE /identity/v3/users/b2d0547b423448e094417e7cca7d67b1 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:15.580825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2adb54-9192-4972-ab74-4e58d6661eb4 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:15.581400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2adb54-9192-4972-ab74-4e58d6661eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:15.581565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2adb54-9192-4972-ab74-4e58d6661eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:15.602067 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-ac2adb54-9192-4972-ab74-4e58d6661eb4 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:15.603385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ac2adb54-9192-4972-ab74-4e58d6661eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:15.666395 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 931/1873] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:15.676758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b93d4-1555-4fbf-b463-bf4e83e5c600 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:15.678269 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5b7b93d4-1555-4fbf-b463-bf4e83e5c600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:15.744593 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b93d4-1555-4fbf-b463-bf4e83e5c600 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:15.752702 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ca6bc-8ade-4230-94c0-2049c5f9c509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:15.753420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ca6bc-8ade-4230-94c0-2049c5f9c509 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:15.753706 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ca6bc-8ade-4230-94c0-2049c5f9c509 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:15.753980 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ca6bc-8ade-4230-94c0-2049c5f9c509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:15.767374 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ca6bc-8ade-4230-94c0-2049c5f9c509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:15.768968 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ca6bc-8ade-4230-94c0-2049c5f9c509 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:15.802087 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 943/1874] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:16.269439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2adb54-9192-4972-ab74-4e58d6661eb4 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:16.271045 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ac2adb54-9192-4972-ab74-4e58d6661eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:16.286955 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2adb54-9192-4972-ab74-4e58d6661eb4 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:16.292922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abffb4-194a-42cd-9ee0-0ec6507e7ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:16.293687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abffb4-194a-42cd-9ee0-0ec6507e7ee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:16.293967 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abffb4-194a-42cd-9ee0-0ec6507e7ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:16.294236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abffb4-194a-42cd-9ee0-0ec6507e7ee4 admin admin] PATH_INFO: `/v3/projects/7a04faeffed14d078bffa686d5a6aaf7` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:16.295494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abffb4-194a-42cd-9ee0-0ec6507e7ee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a04faeffed14d078bffa686d5a6aaf7)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:16.298221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abffb4-194a-42cd-9ee0-0ec6507e7ee4 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:16.396642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ca6bc-8ade-4230-94c0-2049c5f9c509 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:16.398246 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4c5ca6bc-8ade-4230-94c0-2049c5f9c509 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:16.425838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:16.450219 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b3abffb4-194a-42cd-9ee0-0ec6507e7ee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a04faeffed14d078bffa686d5a6aaf7', 'request_id': 'req-b3abffb4-194a-42cd-9ee0-0ec6507e7ee4'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:16.450219 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b3abffb4-194a-42cd-9ee0-0ec6507e7ee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a04faeffed14d078bffa686d5a6aaf7', 'request_id': 'req-b3abffb4-194a-42cd-9ee0-0ec6507e7ee4'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:16.453454 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b3abffb4-194a-42cd-9ee0-0ec6507e7ee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a04faeffed14d078bffa686d5a6aaf7', 'request_id': 'req-b3abffb4-194a-42cd-9ee0-0ec6507e7ee4'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:16.461355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ca6bc-8ade-4230-94c0-2049c5f9c509 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:16.510210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda28711-ae6b-4511-84d8-e4118d848a50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:16.510898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda28711-ae6b-4511-84d8-e4118d848a50 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:16.511181 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda28711-ae6b-4511-84d8-e4118d848a50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:16.511456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda28711-ae6b-4511-84d8-e4118d848a50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:16.513222 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-cda28711-ae6b-4511-84d8-e4118d848a50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:16.531057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b3abffb4-194a-42cd-9ee0-0ec6507e7ee4 admin admin] The token cache is being invalidate because project 7a04faeffed14d078bffa686d5a6aaf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:16.531440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b3abffb4-194a-42cd-9ee0-0ec6507e7ee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:16.532260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b3abffb4-194a-42cd-9ee0-0ec6507e7ee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:16.533706 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 932/1875] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:16 2022] DELETE /identity/v3/projects/7a04faeffed14d078bffa686d5a6aaf7 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:16.538604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616cdd2-81a1-41c4-975a-c3d0a57677b2 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:16.538896 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616cdd2-81a1-41c4-975a-c3d0a57677b2 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:16.539172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616cdd2-81a1-41c4-975a-c3d0a57677b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:16.559686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-0616cdd2-81a1-41c4-975a-c3d0a57677b2 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:16.561058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-0616cdd2-81a1-41c4-975a-c3d0a57677b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:16.581568 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda28711-ae6b-4511-84d8-e4118d848a50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:16.583034 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda28711-ae6b-4511-84d8-e4118d848a50 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:16.594149 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 944/1876] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:16.639233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 933/1877] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:16.646093 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892a56c-9454-4654-b31f-67c2f9939d24 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:16.646385 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892a56c-9454-4654-b31f-67c2f9939d24 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:16.646665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892a56c-9454-4654-b31f-67c2f9939d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:16.670824 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-2892a56c-9454-4654-b31f-67c2f9939d24 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:16.672263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2892a56c-9454-4654-b31f-67c2f9939d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:16.681480 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 945/1878] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:16.685907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616cdd2-81a1-41c4-975a-c3d0a57677b2 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:16.700683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616cdd2-81a1-41c4-975a-c3d0a57677b2 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:16.711989 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37f279-a6c9-4619-a266-661347794ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:16.715978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37f279-a6c9-4619-a266-661347794ffd admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:16.715978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37f279-a6c9-4619-a266-661347794ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:16.715978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37f279-a6c9-4619-a266-661347794ffd admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:16.715978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37f279-a6c9-4619-a266-661347794ffd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:49:16.715978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37f279-a6c9-4619-a266-661347794ffd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:16.717148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37f279-a6c9-4619-a266-661347794ffd admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:16.731098 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 934/1879] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:49:16.735713 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892a56c-9454-4654-b31f-67c2f9939d24 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:16.746201 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892a56c-9454-4654-b31f-67c2f9939d24 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:16.751537 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1b1bf-1648-4dea-ae67-193a75fc4a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:16.752264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1b1bf-1648-4dea-ae67-193a75fc4a01 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:16.752698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1b1bf-1648-4dea-ae67-193a75fc4a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:16.753039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1b1bf-1648-4dea-ae67-193a75fc4a01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:16.753825 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1b1bf-1648-4dea-ae67-193a75fc4a01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:16.755869 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1b1bf-1648-4dea-ae67-193a75fc4a01 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:16.865297 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 946/1880] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:16 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:16.870258 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37f279-a6c9-4619-a266-661347794ffd admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:16.881893 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37f279-a6c9-4619-a266-661347794ffd admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:16.891791 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:16.965689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1b1bf-1648-4dea-ae67-193a75fc4a01 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:16.968186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a9a1b1bf-1648-4dea-ae67-193a75fc4a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:16.981836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7fd24-0b5d-4986-9f0a-245c812f196a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:16.983141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7fd24-0b5d-4986-9f0a-245c812f196a admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:16.983597 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7fd24-0b5d-4986-9f0a-245c812f196a admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:16.984009 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7fd24-0b5d-4986-9f0a-245c812f196a admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:16.984691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f7fd24-0b5d-4986-9f0a-245c812f196a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:16.987967 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f7fd24-0b5d-4986-9f0a-245c812f196a admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:17.075351 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1b1bf-1648-4dea-ae67-193a75fc4a01 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:17.080244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b031b8b-210e-4da9-b7a0-acb6ac3a7273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:17.081031 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b031b8b-210e-4da9-b7a0-acb6ac3a7273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:17.081311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b031b8b-210e-4da9-b7a0-acb6ac3a7273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:17.081661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b031b8b-210e-4da9-b7a0-acb6ac3a7273 admin admin] PATH_INFO: `/v3/users/0706c9a5c3294a898a4b452c872449ed` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:17.082955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b031b8b-210e-4da9-b7a0-acb6ac3a7273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0706c9a5c3294a898a4b452c872449ed)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:17.085177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b031b8b-210e-4da9-b7a0-acb6ac3a7273 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:17.087601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:17.165738 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 935/1881] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:16 2022] POST /identity/v3/users => generated 292 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:17.167896 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7fd24-0b5d-4986-9f0a-245c812f196a admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:17.178081 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7fd24-0b5d-4986-9f0a-245c812f196a admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:17.183404 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f6bb3-483f-4f77-890c-a5ef56f95f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:17.184013 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f6bb3-483f-4f77-890c-a5ef56f95f6c admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:17.184295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f6bb3-483f-4f77-890c-a5ef56f95f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:17.184627 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f6bb3-483f-4f77-890c-a5ef56f95f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:17.185269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f6bb3-483f-4f77-890c-a5ef56f95f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:17.186886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f6bb3-483f-4f77-890c-a5ef56f95f6c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:17.198537 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 936/1882] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:49:17.214433 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f6bb3-483f-4f77-890c-a5ef56f95f6c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:17.224040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f6bb3-483f-4f77-890c-a5ef56f95f6c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:17.228798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff80f8-f06e-4961-ad5c-20331729356e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:17.229398 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff80f8-f06e-4961-ad5c-20331729356e admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:17.229674 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff80f8-f06e-4961-ad5c-20331729356e admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:17.229942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff80f8-f06e-4961-ad5c-20331729356e admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:17.231086 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff80f8-f06e-4961-ad5c-20331729356e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:17.232391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff80f8-f06e-4961-ad5c-20331729356e admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:17.246882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 937/1883] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:49:17.247808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff80f8-f06e-4961-ad5c-20331729356e admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:17.259597 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff80f8-f06e-4961-ad5c-20331729356e admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:17.272922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cea1d1-fc33-4f3f-886b-043cca6b799d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:17.272922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cea1d1-fc33-4f3f-886b-043cca6b799d admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:17.272922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cea1d1-fc33-4f3f-886b-043cca6b799d admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:17.272922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cea1d1-fc33-4f3f-886b-043cca6b799d admin admin] PATH_INFO: `/v3/projects/319820259eeb4b56b01b7698570ceff7/users/1715248c7cd84dc982e8e56d726edb2c/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:17.286587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cea1d1-fc33-4f3f-886b-043cca6b799d admin admin] RBAC: Authorizing `identity:create_grant(project_id=319820259eeb4b56b01b7698570ceff7, user_id=1715248c7cd84dc982e8e56d726edb2c, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:17.289340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cea1d1-fc33-4f3f-886b-043cca6b799d admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:17.377168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 938/1884] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:17 2022] PUT /identity/v3/projects/319820259eeb4b56b01b7698570ceff7/users/1715248c7cd84dc982e8e56d726edb2c/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:17.387756 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd427496-d273-4a1b-ba2d-2552c3cbe9a0 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:17.388163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd427496-d273-4a1b-ba2d-2552c3cbe9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:17.388545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd427496-d273-4a1b-ba2d-2552c3cbe9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:17.396636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3b031b8b-210e-4da9-b7a0-acb6ac3a7273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0706c9a5c3294a898a4b452c872449ed'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:17.398283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3b031b8b-210e-4da9-b7a0-acb6ac3a7273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0706c9a5c3294a898a4b452c872449ed'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:17.399179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3b031b8b-210e-4da9-b7a0-acb6ac3a7273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0706c9a5c3294a898a4b452c872449ed'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:17.424994 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-dd427496-d273-4a1b-ba2d-2552c3cbe9a0 None None] MFA Rules not processed for user `1715248c7cd84dc982e8e56d726edb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:17.426536 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-dd427496-d273-4a1b-ba2d-2552c3cbe9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:17.493228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 939/1885] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:17.496542 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3b031b8b-210e-4da9-b7a0-acb6ac3a7273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0706c9a5c3294a898a4b452c872449ed'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:17.498722 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd427496-d273-4a1b-ba2d-2552c3cbe9a0 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:17.500066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-dd427496-d273-4a1b-ba2d-2552c3cbe9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:17.508379 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3b031b8b-210e-4da9-b7a0-acb6ac3a7273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0706c9a5c3294a898a4b452c872449ed'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:17.534123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 947/1886] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:16 2022] DELETE /identity/v3/users/0706c9a5c3294a898a4b452c872449ed => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:17.538833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38969ded-b477-4252-841f-8b450d6289fc None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:17.539121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38969ded-b477-4252-841f-8b450d6289fc None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:17.539404 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38969ded-b477-4252-841f-8b450d6289fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:17.560022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-38969ded-b477-4252-841f-8b450d6289fc None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:17.561288 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-38969ded-b477-4252-841f-8b450d6289fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:17.561472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:17.581595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd427496-d273-4a1b-ba2d-2552c3cbe9a0 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:17.632842 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 948/1887] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:17.640523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef9ac9-49bd-4b4f-a5df-3e94e0c6a4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:17.641160 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef9ac9-49bd-4b4f-a5df-3e94e0c6a4e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:17.641511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef9ac9-49bd-4b4f-a5df-3e94e0c6a4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:17.641871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef9ac9-49bd-4b4f-a5df-3e94e0c6a4e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:17.642504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef9ac9-49bd-4b4f-a5df-3e94e0c6a4e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:17.643062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38969ded-b477-4252-841f-8b450d6289fc None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:17.644437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef9ac9-49bd-4b4f-a5df-3e94e0c6a4e7 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:17.644624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-38969ded-b477-4252-841f-8b450d6289fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:17.707592 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38969ded-b477-4252-841f-8b450d6289fc None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:17.712288 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4539daa-0a80-4c16-b7da-6da2a7da494f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:17.713063 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4539daa-0a80-4c16-b7da-6da2a7da494f service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:17.714033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4539daa-0a80-4c16-b7da-6da2a7da494f service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:17.717124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4539daa-0a80-4c16-b7da-6da2a7da494f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:17.724969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4539daa-0a80-4c16-b7da-6da2a7da494f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:17.726620 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4539daa-0a80-4c16-b7da-6da2a7da494f service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:17.745986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 940/1889] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:17 2022] POST /identity/v3/projects => generated 383 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:17.748440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 949/1889] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:17.750130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef9ac9-49bd-4b4f-a5df-3e94e0c6a4e7 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:17.841376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef9ac9-49bd-4b4f-a5df-3e94e0c6a4e7 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:17.848929 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75c6b8-3168-4e23-9dee-5e47459b579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:17.850112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75c6b8-3168-4e23-9dee-5e47459b579e admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:17.850112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75c6b8-3168-4e23-9dee-5e47459b579e admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:17.852165 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75c6b8-3168-4e23-9dee-5e47459b579e admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:17.852165 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75c6b8-3168-4e23-9dee-5e47459b579e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:17.853776 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75c6b8-3168-4e23-9dee-5e47459b579e admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:17.978555 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 941/1890] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:17 2022] POST /identity/v3/users => generated 292 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:17.982982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4539daa-0a80-4c16-b7da-6da2a7da494f service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:17.994131 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4539daa-0a80-4c16-b7da-6da2a7da494f service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:17.998927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c2058-b873-40c7-aa0f-0842a7b5ef82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:17.999757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c2058-b873-40c7-aa0f-0842a7b5ef82 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:17.999963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c2058-b873-40c7-aa0f-0842a7b5ef82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:18.000233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c2058-b873-40c7-aa0f-0842a7b5ef82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:18.000983 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c2058-b873-40c7-aa0f-0842a7b5ef82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:18.002835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c2058-b873-40c7-aa0f-0842a7b5ef82 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:18.018389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75c6b8-3168-4e23-9dee-5e47459b579e admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:18.025105 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 950/1891] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:18.030476 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75c6b8-3168-4e23-9dee-5e47459b579e admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:18.037010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0b02e-a153-4b4e-86b1-59cbcdbed92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:18.037861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0b02e-a153-4b4e-86b1-59cbcdbed92b admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:18.038321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0b02e-a153-4b4e-86b1-59cbcdbed92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:18.038958 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0b02e-a153-4b4e-86b1-59cbcdbed92b admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:18.039812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0b02e-a153-4b4e-86b1-59cbcdbed92b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:18.042437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0b02e-a153-4b4e-86b1-59cbcdbed92b admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:18.058312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 942/1892] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:18.061327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c2058-b873-40c7-aa0f-0842a7b5ef82 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:18.076152 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c2058-b873-40c7-aa0f-0842a7b5ef82 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:18.082264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efda1fa-05e3-4aa0-aa14-6fc9fdcb7767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:18.083521 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efda1fa-05e3-4aa0-aa14-6fc9fdcb7767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:18.083968 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efda1fa-05e3-4aa0-aa14-6fc9fdcb7767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:18.084369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efda1fa-05e3-4aa0-aa14-6fc9fdcb7767 admin admin] PATH_INFO: `/v3/projects/5de0589689b4485e9168630e90f5f2b3/users/33bd36b761a44df99f46e4fea9d991d3/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:18.103196 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:18.120564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efda1fa-05e3-4aa0-aa14-6fc9fdcb7767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de0589689b4485e9168630e90f5f2b3, user_id=33bd36b761a44df99f46e4fea9d991d3, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:18.125206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efda1fa-05e3-4aa0-aa14-6fc9fdcb7767 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:18.198328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 951/1893] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:18 2022] PUT /identity/v3/projects/5de0589689b4485e9168630e90f5f2b3/users/33bd36b761a44df99f46e4fea9d991d3/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:18.210229 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3bba8-0739-478d-9d02-9503a2790f64 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:18.210570 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3bba8-0739-478d-9d02-9503a2790f64 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:18.210856 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3bba8-0739-478d-9d02-9503a2790f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:18.241978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-51d3bba8-0739-478d-9d02-9503a2790f64 None None] MFA Rules not processed for user `33bd36b761a44df99f46e4fea9d991d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:18.243700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-51d3bba8-0739-478d-9d02-9503a2790f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:18.295101 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:18.317108 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 943/1894] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:18.321734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efda1fa-05e3-4aa0-aa14-6fc9fdcb7767 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:18.402260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3bba8-0739-478d-9d02-9503a2790f64 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:18.404535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-51d3bba8-0739-478d-9d02-9503a2790f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:18.408531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efda1fa-05e3-4aa0-aa14-6fc9fdcb7767 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:18.415575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd848ea-b418-4eeb-b63e-892a50a73c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:18.416312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd848ea-b418-4eeb-b63e-892a50a73c58 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:18.417272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd848ea-b418-4eeb-b63e-892a50a73c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:18.417652 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd848ea-b418-4eeb-b63e-892a50a73c58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:18.418426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd848ea-b418-4eeb-b63e-892a50a73c58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:18.420483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd848ea-b418-4eeb-b63e-892a50a73c58 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:18.423866 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3bba8-0739-478d-9d02-9503a2790f64 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:18.431028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1d45d-0476-46dc-95bc-3804e837f115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:18.431949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1d45d-0476-46dc-95bc-3804e837f115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:18.432769 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1d45d-0476-46dc-95bc-3804e837f115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:18.432999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1d45d-0476-46dc-95bc-3804e837f115 admin admin] PATH_INFO: `/v3/projects/c82aef4f789e49b8930cc0410a5dc80c` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:18.434252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1d45d-0476-46dc-95bc-3804e837f115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c82aef4f789e49b8930cc0410a5dc80c)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:18.436520 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1d45d-0476-46dc-95bc-3804e837f115 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:18.528863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 952/1895] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:18 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:18.530835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd848ea-b418-4eeb-b63e-892a50a73c58 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:18.590072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-5fa1d45d-0476-46dc-95bc-3804e837f115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c82aef4f789e49b8930cc0410a5dc80c', 'request_id': 'req-5fa1d45d-0476-46dc-95bc-3804e837f115'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:18.597796 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-5fa1d45d-0476-46dc-95bc-3804e837f115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c82aef4f789e49b8930cc0410a5dc80c', 'request_id': 'req-5fa1d45d-0476-46dc-95bc-3804e837f115'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:18.605087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-5fa1d45d-0476-46dc-95bc-3804e837f115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c82aef4f789e49b8930cc0410a5dc80c', 'request_id': 'req-5fa1d45d-0476-46dc-95bc-3804e837f115'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:18.633062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd848ea-b418-4eeb-b63e-892a50a73c58 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:18.672701 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-5fa1d45d-0476-46dc-95bc-3804e837f115 admin admin] The token cache is being invalidate because project c82aef4f789e49b8930cc0410a5dc80c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:18.673291 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-5fa1d45d-0476-46dc-95bc-3804e837f115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:18.673818 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-5fa1d45d-0476-46dc-95bc-3804e837f115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:18.675343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 944/1896] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:18 2022] DELETE /identity/v3/projects/c82aef4f789e49b8930cc0410a5dc80c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:18.676184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1d45d-0476-46dc-95bc-3804e837f115 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:18.677468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-5fa1d45d-0476-46dc-95bc-3804e837f115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:18.693213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6cf680-c53d-4bd3-96ac-5c3ca0510584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:18.693828 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cf680-c53d-4bd3-96ac-5c3ca0510584 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:18.694113 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cf680-c53d-4bd3-96ac-5c3ca0510584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:18.694418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cf680-c53d-4bd3-96ac-5c3ca0510584 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:18.699871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6cf680-c53d-4bd3-96ac-5c3ca0510584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:18.699871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6cf680-c53d-4bd3-96ac-5c3ca0510584 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:18.742975 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1d45d-0476-46dc-95bc-3804e837f115 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:18.747672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad488638-e33c-4b5b-9821-4c9ee5364fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:18.748304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad488638-e33c-4b5b-9821-4c9ee5364fe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:18.748620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad488638-e33c-4b5b-9821-4c9ee5364fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:18.748897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad488638-e33c-4b5b-9821-4c9ee5364fe5 admin admin] PATH_INFO: `/v3/users/af29bb9fd4974e64908c696620065dde` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:18.750026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad488638-e33c-4b5b-9821-4c9ee5364fe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af29bb9fd4974e64908c696620065dde)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:18.751995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad488638-e33c-4b5b-9821-4c9ee5364fe5 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:18.798955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 953/1897] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:18 2022] POST /identity/v3/users => generated 299 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:18.800300 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a030d14-92ef-4071-a212-03ac08fc2264 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:18.800617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a030d14-92ef-4071-a212-03ac08fc2264 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:18.800894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a030d14-92ef-4071-a212-03ac08fc2264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:18.819764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-2a030d14-92ef-4071-a212-03ac08fc2264 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:18.821000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2a030d14-92ef-4071-a212-03ac08fc2264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:18.828767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 954/1898] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:18.829658 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a030d14-92ef-4071-a212-03ac08fc2264 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:18.830856 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2a030d14-92ef-4071-a212-03ac08fc2264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:18.837475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:18.842965 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a030d14-92ef-4071-a212-03ac08fc2264 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:18.847479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ead1f-23e8-415d-9c55-a17c3086e88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:18.849047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ead1f-23e8-415d-9c55-a17c3086e88c admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:18.849047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ead1f-23e8-415d-9c55-a17c3086e88c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:18.849047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ead1f-23e8-415d-9c55-a17c3086e88c admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:18.849309 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ead1f-23e8-415d-9c55-a17c3086e88c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:18.850784 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ead1f-23e8-415d-9c55-a17c3086e88c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:18.860875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 955/1899] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:18.862147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcaa4b-b30f-4075-82db-0a11816b6d81 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:18.862437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcaa4b-b30f-4075-82db-0a11816b6d81 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:18.862715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcaa4b-b30f-4075-82db-0a11816b6d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:18.881398 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-befcaa4b-b30f-4075-82db-0a11816b6d81 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:18.882585 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-befcaa4b-b30f-4075-82db-0a11816b6d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:18.890862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 956/1900] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:18.891735 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befcaa4b-b30f-4075-82db-0a11816b6d81 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:18.900607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befcaa4b-b30f-4075-82db-0a11816b6d81 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:18.905672 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431b886-b1dd-488d-9e4a-b584d108f9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:18.906287 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431b886-b1dd-488d-9e4a-b584d108f9fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:18.906571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431b886-b1dd-488d-9e4a-b584d108f9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:18.906846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431b886-b1dd-488d-9e4a-b584d108f9fd admin admin] PATH_INFO: `/v3/projects/56f05d774edb4917bb4926dc03516d69/users/4e69211197d1412caa2799b16676fef6/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:18.920908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431b886-b1dd-488d-9e4a-b584d108f9fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f05d774edb4917bb4926dc03516d69, user_id=4e69211197d1412caa2799b16676fef6, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:18.923551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431b886-b1dd-488d-9e4a-b584d108f9fd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:18.926523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad488638-e33c-4b5b-9821-4c9ee5364fe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af29bb9fd4974e64908c696620065dde'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:18.931768 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad488638-e33c-4b5b-9821-4c9ee5364fe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af29bb9fd4974e64908c696620065dde'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:18.931768 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad488638-e33c-4b5b-9821-4c9ee5364fe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af29bb9fd4974e64908c696620065dde'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:18.934816 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad488638-e33c-4b5b-9821-4c9ee5364fe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af29bb9fd4974e64908c696620065dde'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:19.001781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad488638-e33c-4b5b-9821-4c9ee5364fe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af29bb9fd4974e64908c696620065dde'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:19.011868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 957/1901] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:18 2022] PUT /identity/v3/projects/56f05d774edb4917bb4926dc03516d69/users/4e69211197d1412caa2799b16676fef6/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:19.011868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431b886-b1dd-488d-9e4a-b584d108f9fd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:19.013265 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:19.013917 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8431b886-b1dd-488d-9e4a-b584d108f9fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:19.024384 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 945/1902] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:18 2022] DELETE /identity/v3/users/af29bb9fd4974e64908c696620065dde => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:19.025079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad488638-e33c-4b5b-9821-4c9ee5364fe5 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:19.027831 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ad488638-e33c-4b5b-9821-4c9ee5364fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:19.089330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431b886-b1dd-488d-9e4a-b584d108f9fd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:19.101376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad488638-e33c-4b5b-9821-4c9ee5364fe5 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:19.106375 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61dc0f-2e5d-4a78-b8dd-d7c7a670e2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:19.107091 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61dc0f-2e5d-4a78-b8dd-d7c7a670e2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:19.107380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61dc0f-2e5d-4a78-b8dd-d7c7a670e2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:19.107654 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61dc0f-2e5d-4a78-b8dd-d7c7a670e2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:19.108299 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61dc0f-2e5d-4a78-b8dd-d7c7a670e2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:19.110161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61dc0f-2e5d-4a78-b8dd-d7c7a670e2c3 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:19.120734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 946/1903] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:19.122132 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a000d-2cd3-41b2-9b9a-06c60257a0cb None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:19.122419 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a000d-2cd3-41b2-9b9a-06c60257a0cb None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:19.122780 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a000d-2cd3-41b2-9b9a-06c60257a0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:19.143329 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-521a000d-2cd3-41b2-9b9a-06c60257a0cb None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:19.143516 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-521a000d-2cd3-41b2-9b9a-06c60257a0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:19.155635 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 947/1904] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:19.156625 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a000d-2cd3-41b2-9b9a-06c60257a0cb None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:19.164831 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86a128-1ae7-4da4-8c59-ac1151e41545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:19.165486 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86a128-1ae7-4da4-8c59-ac1151e41545 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:19.167447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86a128-1ae7-4da4-8c59-ac1151e41545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:19.167447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86a128-1ae7-4da4-8c59-ac1151e41545 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:19.167633 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a000d-2cd3-41b2-9b9a-06c60257a0cb None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:19.167723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86a128-1ae7-4da4-8c59-ac1151e41545 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:49:19.167723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86a128-1ae7-4da4-8c59-ac1151e41545 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:19.168719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86a128-1ae7-4da4-8c59-ac1151e41545 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:19.171531 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182122d-67f3-4ede-8b34-a201a7b112aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:19.172185 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182122d-67f3-4ede-8b34-a201a7b112aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:19.172498 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182122d-67f3-4ede-8b34-a201a7b112aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:19.172777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182122d-67f3-4ede-8b34-a201a7b112aa admin admin] PATH_INFO: `/v3/projects/56f05d774edb4917bb4926dc03516d69/users/4e69211197d1412caa2799b16676fef6/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:19.174716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182122d-67f3-4ede-8b34-a201a7b112aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f05d774edb4917bb4926dc03516d69, user_id=4e69211197d1412caa2799b16676fef6, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:19.177419 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182122d-67f3-4ede-8b34-a201a7b112aa admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:19.180775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 958/1905] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:49:19.184279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86a128-1ae7-4da4-8c59-ac1151e41545 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:19.185910 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ac86a128-1ae7-4da4-8c59-ac1151e41545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:19.188980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 948/1906] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:19 2022] PUT /identity/v3/projects/56f05d774edb4917bb4926dc03516d69/users/4e69211197d1412caa2799b16676fef6/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:19.194914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182122d-67f3-4ede-8b34-a201a7b112aa admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:19.250548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86a128-1ae7-4da4-8c59-ac1151e41545 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:19.261567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182122d-67f3-4ede-8b34-a201a7b112aa admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:19.263263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:19.266700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b9797-7e68-4bf7-9053-8fbf85047855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:19.267304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b9797-7e68-4bf7-9053-8fbf85047855 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:19.267586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b9797-7e68-4bf7-9053-8fbf85047855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:19.267857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b9797-7e68-4bf7-9053-8fbf85047855 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:19.268519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b9797-7e68-4bf7-9053-8fbf85047855 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:19.270400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b9797-7e68-4bf7-9053-8fbf85047855 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:19.299548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706cb22-d477-446c-88a5-583e6d492286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:19.300170 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706cb22-d477-446c-88a5-583e6d492286 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:19.300476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706cb22-d477-446c-88a5-583e6d492286 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:19.300754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706cb22-d477-446c-88a5-583e6d492286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:19.310303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706cb22-d477-446c-88a5-583e6d492286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:19.311847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706cb22-d477-446c-88a5-583e6d492286 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:19.321118 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 959/1907] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:19.322275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b15fe3-b600-4624-bba3-1999aaec8ff5 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:19.323232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b15fe3-b600-4624-bba3-1999aaec8ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:19.323520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b15fe3-b600-4624-bba3-1999aaec8ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:19.352355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e9b15fe3-b600-4624-bba3-1999aaec8ff5 None None] MFA Rules not processed for user `4e69211197d1412caa2799b16676fef6`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:19.353605 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e9b15fe3-b600-4624-bba3-1999aaec8ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:19.402648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 949/1908] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:19 2022] POST /identity/v3/projects => generated 373 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:19.403606 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b9797-7e68-4bf7-9053-8fbf85047855 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:19.476963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 960/1909] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:19.485495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b15fe3-b600-4624-bba3-1999aaec8ff5 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:19.488945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e9b15fe3-b600-4624-bba3-1999aaec8ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:19.508223 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:19.531863 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b9797-7e68-4bf7-9053-8fbf85047855 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:19.544648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c38ccd-529b-4fa7-8753-17793436cb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:19.545478 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c38ccd-529b-4fa7-8753-17793436cb9d admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:19.546114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c38ccd-529b-4fa7-8753-17793436cb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:19.546114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c38ccd-529b-4fa7-8753-17793436cb9d admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:19.547336 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c38ccd-529b-4fa7-8753-17793436cb9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:19.551553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c38ccd-529b-4fa7-8753-17793436cb9d admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:19.557289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b15fe3-b600-4624-bba3-1999aaec8ff5 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:19.625498 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddfcf9-e1a5-4373-9b09-cea5b5e51209 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:19.626257 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddfcf9-e1a5-4373-9b09-cea5b5e51209 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:19.626544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddfcf9-e1a5-4373-9b09-cea5b5e51209 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:19.626841 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddfcf9-e1a5-4373-9b09-cea5b5e51209 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:19.628800 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e5ddfcf9-e1a5-4373-9b09-cea5b5e51209 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:19.647495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:19.655766 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 950/1910] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:19 2022] POST /identity/v3/users => generated 287 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:19.659378 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c38ccd-529b-4fa7-8753-17793436cb9d admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:19.675792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c38ccd-529b-4fa7-8753-17793436cb9d admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:19.709119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e14f1b-7067-4def-b3b2-7eb6a5a3b942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:19.709119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e14f1b-7067-4def-b3b2-7eb6a5a3b942 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:19.709119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e14f1b-7067-4def-b3b2-7eb6a5a3b942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:19.709119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e14f1b-7067-4def-b3b2-7eb6a5a3b942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:19.709471 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e14f1b-7067-4def-b3b2-7eb6a5a3b942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:19.709770 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e14f1b-7067-4def-b3b2-7eb6a5a3b942 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:19.725174 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 951/1911] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:49:19.729648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e14f1b-7067-4def-b3b2-7eb6a5a3b942 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:19.743887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e14f1b-7067-4def-b3b2-7eb6a5a3b942 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:19.751032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa1e35-5163-41a6-880a-562b32b0dffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:19.752339 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa1e35-5163-41a6-880a-562b32b0dffb admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:19.752554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa1e35-5163-41a6-880a-562b32b0dffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:19.752901 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa1e35-5163-41a6-880a-562b32b0dffb admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:19.754167 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa1e35-5163-41a6-880a-562b32b0dffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:19.756523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa1e35-5163-41a6-880a-562b32b0dffb admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:19.764990 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddfcf9-e1a5-4373-9b09-cea5b5e51209 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:19.766947 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddfcf9-e1a5-4373-9b09-cea5b5e51209 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:19.771397 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 952/1912] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:19.775232 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa1e35-5163-41a6-880a-562b32b0dffb admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:19.800088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 961/1913] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:19.805337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa1e35-5163-41a6-880a-562b32b0dffb admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:19.821687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bb7e1-6ae5-4c23-b291-770e63db805f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:19.823240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bb7e1-6ae5-4c23-b291-770e63db805f admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:19.823528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bb7e1-6ae5-4c23-b291-770e63db805f admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:19.823804 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bb7e1-6ae5-4c23-b291-770e63db805f admin admin] PATH_INFO: `/v3/projects/cd5a0fa1909747368101df31693c44e1/users/55975de527254dca90fc4f30bc50e393/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:19.857933 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4bb7e1-6ae5-4c23-b291-770e63db805f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd5a0fa1909747368101df31693c44e1, user_id=55975de527254dca90fc4f30bc50e393, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:19.861353 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4bb7e1-6ae5-4c23-b291-770e63db805f admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:19.975764 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 953/1914] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:19 2022] PUT /identity/v3/projects/cd5a0fa1909747368101df31693c44e1/users/55975de527254dca90fc4f30bc50e393/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:19.982893 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89277fd7-d2af-46e0-a666-392e49ab3808 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:19.983347 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89277fd7-d2af-46e0-a666-392e49ab3808 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:19.983729 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89277fd7-d2af-46e0-a666-392e49ab3808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:20.008477 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-89277fd7-d2af-46e0-a666-392e49ab3808 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:20.009591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-89277fd7-d2af-46e0-a666-392e49ab3808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:20.060422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bb7e1-6ae5-4c23-b291-770e63db805f admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:20.068212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1b4bb7e1-6ae5-4c23-b291-770e63db805f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:20.100279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 962/1915] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:20.112637 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89277fd7-d2af-46e0-a666-392e49ab3808 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:20.114508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-89277fd7-d2af-46e0-a666-392e49ab3808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:20.172391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bb7e1-6ae5-4c23-b291-770e63db805f admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:20.197081 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:20.199084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:20.199432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:20.199707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365 admin admin] PATH_INFO: `/v3/projects/48651184e30d4c2480d534554351a7df` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:20.202553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48651184e30d4c2480d534554351a7df)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:20.206685 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:20.214385 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89277fd7-d2af-46e0-a666-392e49ab3808 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:20.219183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fd199-7428-4ccb-aeea-d6fab251d304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:20.220451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fd199-7428-4ccb-aeea-d6fab251d304 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:20.223136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fd199-7428-4ccb-aeea-d6fab251d304 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:20.223584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fd199-7428-4ccb-aeea-d6fab251d304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:20.243946 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fd199-7428-4ccb-aeea-d6fab251d304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:20.246057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fd199-7428-4ccb-aeea-d6fab251d304 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:20.257645 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 963/1916] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:49:20.380481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48651184e30d4c2480d534554351a7df', 'request_id': 'req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:20.392028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48651184e30d4c2480d534554351a7df', 'request_id': 'req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:20.412144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48651184e30d4c2480d534554351a7df', 'request_id': 'req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:20.518482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365 admin admin] The token cache is being invalidate because project 48651184e30d4c2480d534554351a7df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:20.518969 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:20.519706 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:20.521708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 954/1917] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:20 2022] DELETE /identity/v3/projects/48651184e30d4c2480d534554351a7df => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:20.526447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fd199-7428-4ccb-aeea-d6fab251d304 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:20.538619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fd199-7428-4ccb-aeea-d6fab251d304 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:20.633429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea425c7-3eda-4460-9f09-6a8c1e329135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:20.634159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea425c7-3eda-4460-9f09-6a8c1e329135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:20.634348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea425c7-3eda-4460-9f09-6a8c1e329135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:20.634623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea425c7-3eda-4460-9f09-6a8c1e329135 admin admin] PATH_INFO: `/v3/users/3c01a452981647908bb4e1ace42d3bff` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:20.637094 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea425c7-3eda-4460-9f09-6a8c1e329135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c01a452981647908bb4e1ace42d3bff)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:20.638792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea425c7-3eda-4460-9f09-6a8c1e329135 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:20.640693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:20.869860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6ea425c7-3eda-4460-9f09-6a8c1e329135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c01a452981647908bb4e1ace42d3bff'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:20.870614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6ea425c7-3eda-4460-9f09-6a8c1e329135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c01a452981647908bb4e1ace42d3bff'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:20.871478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6ea425c7-3eda-4460-9f09-6a8c1e329135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c01a452981647908bb4e1ace42d3bff'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:20.978690 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6ea425c7-3eda-4460-9f09-6a8c1e329135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c01a452981647908bb4e1ace42d3bff'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:20.989072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6ea425c7-3eda-4460-9f09-6a8c1e329135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c01a452981647908bb4e1ace42d3bff'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:21.012007 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 964/1918] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:20 2022] DELETE /identity/v3/users/3c01a452981647908bb4e1ace42d3bff => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:21.612747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:21.612747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:21.626038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3e4ea-7d01-41e1-8dcc-408f452ee365 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:21.651100 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:21.715986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926815b6-fd76-4e0f-99d1-409829d0161e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:21.716657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926815b6-fd76-4e0f-99d1-409829d0161e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:21.716941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926815b6-fd76-4e0f-99d1-409829d0161e admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:21.717220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926815b6-fd76-4e0f-99d1-409829d0161e admin admin] PATH_INFO: `/v3/projects/3f12cf6c0d3f4ef8946e98d2e03f8194` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:21.718352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926815b6-fd76-4e0f-99d1-409829d0161e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f12cf6c0d3f4ef8946e98d2e03f8194)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:21.720256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926815b6-fd76-4e0f-99d1-409829d0161e admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:21.783789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0244a-2519-4c5d-947f-62e9a8fbcef1 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:21.784217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0244a-2519-4c5d-947f-62e9a8fbcef1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:21.784217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0244a-2519-4c5d-947f-62e9a8fbcef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:21.814054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e9f0244a-2519-4c5d-947f-62e9a8fbcef1 None None] MFA Rules not processed for user `55975de527254dca90fc4f30bc50e393`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:21.815496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e9f0244a-2519-4c5d-947f-62e9a8fbcef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:21.828518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-926815b6-fd76-4e0f-99d1-409829d0161e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f12cf6c0d3f4ef8946e98d2e03f8194', 'request_id': 'req-926815b6-fd76-4e0f-99d1-409829d0161e'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:21.853503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-926815b6-fd76-4e0f-99d1-409829d0161e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f12cf6c0d3f4ef8946e98d2e03f8194', 'request_id': 'req-926815b6-fd76-4e0f-99d1-409829d0161e'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:21.880313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-926815b6-fd76-4e0f-99d1-409829d0161e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f12cf6c0d3f4ef8946e98d2e03f8194', 'request_id': 'req-926815b6-fd76-4e0f-99d1-409829d0161e'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:21.930402 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 965/1919] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:21.941866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0244a-2519-4c5d-947f-62e9a8fbcef1 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:21.943266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e9f0244a-2519-4c5d-947f-62e9a8fbcef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:22.009279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0244a-2519-4c5d-947f-62e9a8fbcef1 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:22.010231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-926815b6-fd76-4e0f-99d1-409829d0161e admin admin] The token cache is being invalidate because project 3f12cf6c0d3f4ef8946e98d2e03f8194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:22.010531 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-926815b6-fd76-4e0f-99d1-409829d0161e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:22.011178 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-926815b6-fd76-4e0f-99d1-409829d0161e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:22.016027 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 955/1920] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:21 2022] DELETE /identity/v3/projects/3f12cf6c0d3f4ef8946e98d2e03f8194 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:22.016164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4163dbf-c34d-4399-97bf-3467bea79945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:22.016164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4163dbf-c34d-4399-97bf-3467bea79945 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:22.016164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4163dbf-c34d-4399-97bf-3467bea79945 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:22.016164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4163dbf-c34d-4399-97bf-3467bea79945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:22.017636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d4163dbf-c34d-4399-97bf-3467bea79945 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:22.017767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7262e0-f30c-47d3-807d-4f3041b67529 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:22.018099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7262e0-f30c-47d3-807d-4f3041b67529 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:22.018386 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7262e0-f30c-47d3-807d-4f3041b67529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:22.032461 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4163dbf-c34d-4399-97bf-3467bea79945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:22.033939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4163dbf-c34d-4399-97bf-3467bea79945 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:22.038747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-2d7262e0-f30c-47d3-807d-4f3041b67529 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:22.040008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2d7262e0-f30c-47d3-807d-4f3041b67529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:22.044235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 966/1921] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:49:22.111005 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 956/1922] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:22.119980 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede50b5e-70f6-43d6-839f-37aa54503160 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:22.120267 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede50b5e-70f6-43d6-839f-37aa54503160 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:22.120570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede50b5e-70f6-43d6-839f-37aa54503160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:22.140015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ede50b5e-70f6-43d6-839f-37aa54503160 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:22.141309 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ede50b5e-70f6-43d6-839f-37aa54503160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:22.156250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 967/1923] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:22.156470 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7262e0-f30c-47d3-807d-4f3041b67529 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:22.164849 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7262e0-f30c-47d3-807d-4f3041b67529 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:22.175536 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b265d-0958-4819-a4fa-ad377a8629cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:22.176390 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b265d-0958-4819-a4fa-ad377a8629cd admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:22.176890 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b265d-0958-4819-a4fa-ad377a8629cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:22.178065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b265d-0958-4819-a4fa-ad377a8629cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:22.179473 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b265d-0958-4819-a4fa-ad377a8629cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:49:22.180083 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b265d-0958-4819-a4fa-ad377a8629cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:22.183582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b265d-0958-4819-a4fa-ad377a8629cd admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:22.196444 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 957/1924] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:49:22.200979 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede50b5e-70f6-43d6-839f-37aa54503160 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:22.206751 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:22.210764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede50b5e-70f6-43d6-839f-37aa54503160 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:22.236874 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f7b83-e086-40b3-b12f-0b3b2c024a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:22.237710 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f7b83-e086-40b3-b12f-0b3b2c024a2e admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:22.237710 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f7b83-e086-40b3-b12f-0b3b2c024a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:22.238251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f7b83-e086-40b3-b12f-0b3b2c024a2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:22.239081 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f7b83-e086-40b3-b12f-0b3b2c024a2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:22.241207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f7b83-e086-40b3-b12f-0b3b2c024a2e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:22.344110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 968/1925] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:22 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:22.344799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b265d-0958-4819-a4fa-ad377a8629cd admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:22.370942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b265d-0958-4819-a4fa-ad377a8629cd admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:22.438383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9d4b6-b12e-4d8c-863f-49a0eeded217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:22.438383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9d4b6-b12e-4d8c-863f-49a0eeded217 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:22.438383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9d4b6-b12e-4d8c-863f-49a0eeded217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:22.438383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9d4b6-b12e-4d8c-863f-49a0eeded217 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:22.438673 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9d4b6-b12e-4d8c-863f-49a0eeded217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:22.438673 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9d4b6-b12e-4d8c-863f-49a0eeded217 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:22.513324 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 958/1926] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:22 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:22.517377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f7b83-e086-40b3-b12f-0b3b2c024a2e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:22.528485 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f7b83-e086-40b3-b12f-0b3b2c024a2e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:22.533117 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45b3fd-e59b-4ea4-8ab5-851bb8caf18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:22.533860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45b3fd-e59b-4ea4-8ab5-851bb8caf18f admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:22.544535 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45b3fd-e59b-4ea4-8ab5-851bb8caf18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:22.544535 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45b3fd-e59b-4ea4-8ab5-851bb8caf18f admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:22.544535 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45b3fd-e59b-4ea4-8ab5-851bb8caf18f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:22.545878 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45b3fd-e59b-4ea4-8ab5-851bb8caf18f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:22.559344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 969/1927] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:49:22.563930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9d4b6-b12e-4d8c-863f-49a0eeded217 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:22.575553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9d4b6-b12e-4d8c-863f-49a0eeded217 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:22.580734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e04ab4-b6e2-4c19-9391-3cbe730b9f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:22.581341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e04ab4-b6e2-4c19-9391-3cbe730b9f55 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:22.581623 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e04ab4-b6e2-4c19-9391-3cbe730b9f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:22.581903 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e04ab4-b6e2-4c19-9391-3cbe730b9f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:22.582546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e04ab4-b6e2-4c19-9391-3cbe730b9f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:22.584133 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e04ab4-b6e2-4c19-9391-3cbe730b9f55 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:22.596370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 959/1928] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:22.604860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45b3fd-e59b-4ea4-8ab5-851bb8caf18f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:22.614929 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45b3fd-e59b-4ea4-8ab5-851bb8caf18f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:22.620948 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3186c4-6985-4862-9b12-edda1a7c4224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:22.621641 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3186c4-6985-4862-9b12-edda1a7c4224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:22.621920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3186c4-6985-4862-9b12-edda1a7c4224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:22.622194 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3186c4-6985-4862-9b12-edda1a7c4224 admin admin] PATH_INFO: `/v3/projects/fa2ae0e1d751435da1bdcacac5b21970/users/86cb9f735ac14e56a948cf89fb6a43b9/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:22.637052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3186c4-6985-4862-9b12-edda1a7c4224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2ae0e1d751435da1bdcacac5b21970, user_id=86cb9f735ac14e56a948cf89fb6a43b9, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:22.639706 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3186c4-6985-4862-9b12-edda1a7c4224 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:22.735587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 970/1929] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:22 2022] PUT /identity/v3/projects/fa2ae0e1d751435da1bdcacac5b21970/users/86cb9f735ac14e56a948cf89fb6a43b9/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:22.746134 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9689bf-22fe-4d1f-be3b-9091284be2a1 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:22.746624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9689bf-22fe-4d1f-be3b-9091284be2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:22.746745 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9689bf-22fe-4d1f-be3b-9091284be2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:22.776927 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-5d9689bf-22fe-4d1f-be3b-9091284be2a1 None None] MFA Rules not processed for user `86cb9f735ac14e56a948cf89fb6a43b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:22.778208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-5d9689bf-22fe-4d1f-be3b-9091284be2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:22.846171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 960/1930] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:22.850519 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3186c4-6985-4862-9b12-edda1a7c4224 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:22.892861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:22.913858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3186c4-6985-4862-9b12-edda1a7c4224 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:22.923707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43794cfc-fd8b-4a95-866c-07e0cb96373c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:22.924383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43794cfc-fd8b-4a95-866c-07e0cb96373c admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:22.924716 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43794cfc-fd8b-4a95-866c-07e0cb96373c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:22.924968 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43794cfc-fd8b-4a95-866c-07e0cb96373c admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:22.925601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43794cfc-fd8b-4a95-866c-07e0cb96373c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:22.927594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43794cfc-fd8b-4a95-866c-07e0cb96373c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:23.022013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 971/1931] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:22 2022] POST /identity/v3/projects => generated 371 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:23.023193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9689bf-22fe-4d1f-be3b-9091284be2a1 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:23.033982 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9689bf-22fe-4d1f-be3b-9091284be2a1 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:23.090841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782f93b-96ef-4766-9ee1-6490f79cdc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:23.091461 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782f93b-96ef-4766-9ee1-6490f79cdc60 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:23.091745 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782f93b-96ef-4766-9ee1-6490f79cdc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:23.092020 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782f93b-96ef-4766-9ee1-6490f79cdc60 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:23.092688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782f93b-96ef-4766-9ee1-6490f79cdc60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:23.094622 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782f93b-96ef-4766-9ee1-6490f79cdc60 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:23.201750 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 961/1932] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:23 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:23.203101 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43794cfc-fd8b-4a95-866c-07e0cb96373c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:23.220213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43794cfc-fd8b-4a95-866c-07e0cb96373c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:23.221882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc89bc0-f62c-48e5-96b6-c740cd7866f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:23.223056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc89bc0-f62c-48e5-96b6-c740cd7866f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:23.223404 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc89bc0-f62c-48e5-96b6-c740cd7866f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:23.223752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc89bc0-f62c-48e5-96b6-c740cd7866f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:23.225181 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc89bc0-f62c-48e5-96b6-c740cd7866f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:23.226391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc89bc0-f62c-48e5-96b6-c740cd7866f9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:23.237788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 972/1933] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:23.242970 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782f93b-96ef-4766-9ee1-6490f79cdc60 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:23.256174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc89bc0-f62c-48e5-96b6-c740cd7866f9 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:23.258101 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ccc89bc0-f62c-48e5-96b6-c740cd7866f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:23.261261 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782f93b-96ef-4766-9ee1-6490f79cdc60 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:23.269675 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e9666-0151-4c60-8d30-bf2dee48f678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:23.270353 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e9666-0151-4c60-8d30-bf2dee48f678 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:23.270659 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e9666-0151-4c60-8d30-bf2dee48f678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:23.270999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e9666-0151-4c60-8d30-bf2dee48f678 admin admin] PATH_INFO: `/v3/projects/a825738e136c404e841788bf92daa81c/users/d3b82f6b26214b7ab35d0e662dc97e96/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:23.288196 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e9666-0151-4c60-8d30-bf2dee48f678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a825738e136c404e841788bf92daa81c, user_id=d3b82f6b26214b7ab35d0e662dc97e96, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:23.291015 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e9666-0151-4c60-8d30-bf2dee48f678 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:23.328577 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc89bc0-f62c-48e5-96b6-c740cd7866f9 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:23.332189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802902c8-a944-4117-bf10-2c543df42cd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:23.332843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802902c8-a944-4117-bf10-2c543df42cd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:23.333124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802902c8-a944-4117-bf10-2c543df42cd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:23.333397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802902c8-a944-4117-bf10-2c543df42cd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:23.335399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-802902c8-a944-4117-bf10-2c543df42cd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:23.370899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 962/1934] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:23 2022] PUT /identity/v3/projects/a825738e136c404e841788bf92daa81c/users/d3b82f6b26214b7ab35d0e662dc97e96/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:23.374939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e9666-0151-4c60-8d30-bf2dee48f678 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:23.399547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802902c8-a944-4117-bf10-2c543df42cd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:23.401155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802902c8-a944-4117-bf10-2c543df42cd0 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:23.412916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 973/1935] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:23.461157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e9666-0151-4c60-8d30-bf2dee48f678 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:23.467793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8f895-52b0-490f-afac-029cb7d68238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:23.468523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8f895-52b0-490f-afac-029cb7d68238 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:23.468895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8f895-52b0-490f-afac-029cb7d68238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:23.469269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8f895-52b0-490f-afac-029cb7d68238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:23.469978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f8f895-52b0-490f-afac-029cb7d68238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:23.472627 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f8f895-52b0-490f-afac-029cb7d68238 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:23.479707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:23.488582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 963/1936] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:23.490312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802902c8-a944-4117-bf10-2c543df42cd0 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:23.507706 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802902c8-a944-4117-bf10-2c543df42cd0 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:23.512851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acac1c-2442-4ff5-9127-d1710bc8185a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:23.513973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acac1c-2442-4ff5-9127-d1710bc8185a admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:23.514272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acac1c-2442-4ff5-9127-d1710bc8185a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:23.514553 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acac1c-2442-4ff5-9127-d1710bc8185a admin admin] PATH_INFO: `/v3/projects/a825738e136c404e841788bf92daa81c/users/d3b82f6b26214b7ab35d0e662dc97e96/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:23.517050 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acac1c-2442-4ff5-9127-d1710bc8185a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a825738e136c404e841788bf92daa81c, user_id=d3b82f6b26214b7ab35d0e662dc97e96, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:23.520202 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acac1c-2442-4ff5-9127-d1710bc8185a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:23.532377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 974/1937] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:23 2022] PUT /identity/v3/projects/a825738e136c404e841788bf92daa81c/users/d3b82f6b26214b7ab35d0e662dc97e96/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:23.543801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190945d-382c-412a-a100-6098d5860129 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:23.544099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190945d-382c-412a-a100-6098d5860129 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:23.544424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190945d-382c-412a-a100-6098d5860129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:23.583181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-f190945d-382c-412a-a100-6098d5860129 None None] MFA Rules not processed for user `d3b82f6b26214b7ab35d0e662dc97e96`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:23.584389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-f190945d-382c-412a-a100-6098d5860129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:23.661520 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 964/1938] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:23.670203 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acac1c-2442-4ff5-9127-d1710bc8185a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:23.671966 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f5acac1c-2442-4ff5-9127-d1710bc8185a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:23.735082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acac1c-2442-4ff5-9127-d1710bc8185a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:23.741467 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5015b-0269-4850-8b7f-0c8bbe530b1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:23.741467 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5015b-0269-4850-8b7f-0c8bbe530b1c service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:23.741467 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5015b-0269-4850-8b7f-0c8bbe530b1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:23.741820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5015b-0269-4850-8b7f-0c8bbe530b1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:23.763268 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5015b-0269-4850-8b7f-0c8bbe530b1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:23.764927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5015b-0269-4850-8b7f-0c8bbe530b1c service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:23.777233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 975/1939] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:23.910642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:24.026518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1437135-8380-4013-8560-f398c7a52bdc None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:24.026927 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1437135-8380-4013-8560-f398c7a52bdc None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:24.027207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1437135-8380-4013-8560-f398c7a52bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:24.046860 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-d1437135-8380-4013-8560-f398c7a52bdc None None] MFA Rules not processed for user `55975de527254dca90fc4f30bc50e393`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:24.048087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d1437135-8380-4013-8560-f398c7a52bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:24.116927 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 965/1940] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:24.335501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5015b-0269-4850-8b7f-0c8bbe530b1c service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:24.337787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-46c5015b-0269-4850-8b7f-0c8bbe530b1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:24.425144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5015b-0269-4850-8b7f-0c8bbe530b1c service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:24.430046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a2d90-93ec-4787-a1ac-69f0919eaf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:24.430691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2d90-93ec-4787-a1ac-69f0919eaf96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:24.430973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2d90-93ec-4787-a1ac-69f0919eaf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:24.431248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2d90-93ec-4787-a1ac-69f0919eaf96 admin admin] PATH_INFO: `/v3/users/dd903de44365478d99845213d2778406` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:24.433104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a2d90-93ec-4787-a1ac-69f0919eaf96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd903de44365478d99845213d2778406)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:24.435052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a2d90-93ec-4787-a1ac-69f0919eaf96 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:24.641510 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-185a2d90-93ec-4787-a1ac-69f0919eaf96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd903de44365478d99845213d2778406'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:24.642218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-185a2d90-93ec-4787-a1ac-69f0919eaf96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd903de44365478d99845213d2778406'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:24.642892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-185a2d90-93ec-4787-a1ac-69f0919eaf96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd903de44365478d99845213d2778406'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:24.718268 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-185a2d90-93ec-4787-a1ac-69f0919eaf96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd903de44365478d99845213d2778406'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:24.726978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-185a2d90-93ec-4787-a1ac-69f0919eaf96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd903de44365478d99845213d2778406'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:24.732777 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:24.757042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 976/1941] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:24 2022] DELETE /identity/v3/users/dd903de44365478d99845213d2778406 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:24.757135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b2397-bd64-4cc4-84f0-24b2bb7db54d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:24.757135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b2397-bd64-4cc4-84f0-24b2bb7db54d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:24.757135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b2397-bd64-4cc4-84f0-24b2bb7db54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:24.786422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-d10b2397-bd64-4cc4-84f0-24b2bb7db54d None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:24.786598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d10b2397-bd64-4cc4-84f0-24b2bb7db54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:24.843359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:24.871208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 966/1942] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:24.880997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a2d90-93ec-4787-a1ac-69f0919eaf96 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:24.883935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-185a2d90-93ec-4787-a1ac-69f0919eaf96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:24.964500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a2d90-93ec-4787-a1ac-69f0919eaf96 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:24.991604 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f52619-5975-4b21-abea-dab08beb5130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:24.993593 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f52619-5975-4b21-abea-dab08beb5130 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:24.994071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f52619-5975-4b21-abea-dab08beb5130 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:24.994902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f52619-5975-4b21-abea-dab08beb5130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:25.027478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f52619-5975-4b21-abea-dab08beb5130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:25.029010 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f52619-5975-4b21-abea-dab08beb5130 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:25.050458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 977/1943] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:49:25.242295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b2397-bd64-4cc4-84f0-24b2bb7db54d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:25.245206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d10b2397-bd64-4cc4-84f0-24b2bb7db54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:25.324926 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b2397-bd64-4cc4-84f0-24b2bb7db54d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:25.328525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731e9a1-1ca5-46c9-8125-0966822169d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:25.329142 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731e9a1-1ca5-46c9-8125-0966822169d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:25.329425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731e9a1-1ca5-46c9-8125-0966822169d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:25.329699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731e9a1-1ca5-46c9-8125-0966822169d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:25.331516 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7731e9a1-1ca5-46c9-8125-0966822169d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:25.408227 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731e9a1-1ca5-46c9-8125-0966822169d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:25.410015 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731e9a1-1ca5-46c9-8125-0966822169d8 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:25.423288 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 967/1944] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:25.801644 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f52619-5975-4b21-abea-dab08beb5130 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:25.803204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b6f52619-5975-4b21-abea-dab08beb5130 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:25.828989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f52619-5975-4b21-abea-dab08beb5130 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:25.835704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e728f8-c494-4293-8c8f-5dd77eda7630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:25.836653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e728f8-c494-4293-8c8f-5dd77eda7630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:25.837164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e728f8-c494-4293-8c8f-5dd77eda7630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:25.839634 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e728f8-c494-4293-8c8f-5dd77eda7630 admin admin] PATH_INFO: `/v3/projects/c3fb217b6f4e43eabb9cb718b77677ca` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:25.841255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e728f8-c494-4293-8c8f-5dd77eda7630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3fb217b6f4e43eabb9cb718b77677ca)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:25.843912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e728f8-c494-4293-8c8f-5dd77eda7630 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:25.890522 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731e9a1-1ca5-46c9-8125-0966822169d8 service cinder] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:25.892297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7731e9a1-1ca5-46c9-8125-0966822169d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:25.906506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731e9a1-1ca5-46c9-8125-0966822169d8 service cinder] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:25.911325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786a223-af3c-4212-974f-f24a524e801d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:25.912026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786a223-af3c-4212-974f-f24a524e801d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:25.912314 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786a223-af3c-4212-974f-f24a524e801d admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:25.913343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786a223-af3c-4212-974f-f24a524e801d admin admin] PATH_INFO: `/v3/users/86cb9f735ac14e56a948cf89fb6a43b9` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:25.914240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786a223-af3c-4212-974f-f24a524e801d admin admin] RBAC: Authorizing `identity:delete_user(user_id=86cb9f735ac14e56a948cf89fb6a43b9)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:25.916444 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786a223-af3c-4212-974f-f24a524e801d admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:26.039258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d3e728f8-c494-4293-8c8f-5dd77eda7630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3fb217b6f4e43eabb9cb718b77677ca', 'request_id': 'req-d3e728f8-c494-4293-8c8f-5dd77eda7630'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:26.047066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d3e728f8-c494-4293-8c8f-5dd77eda7630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3fb217b6f4e43eabb9cb718b77677ca', 'request_id': 'req-d3e728f8-c494-4293-8c8f-5dd77eda7630'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:26.054164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d3e728f8-c494-4293-8c8f-5dd77eda7630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3fb217b6f4e43eabb9cb718b77677ca', 'request_id': 'req-d3e728f8-c494-4293-8c8f-5dd77eda7630'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:26.168313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d3e728f8-c494-4293-8c8f-5dd77eda7630 admin admin] The token cache is being invalidate because project c3fb217b6f4e43eabb9cb718b77677ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:26.168644 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d3e728f8-c494-4293-8c8f-5dd77eda7630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:26.169464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d3e728f8-c494-4293-8c8f-5dd77eda7630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:26.171272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 978/1945] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:25 2022] DELETE /identity/v3/projects/c3fb217b6f4e43eabb9cb718b77677ca => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:26.176848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e728f8-c494-4293-8c8f-5dd77eda7630 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:26.179686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d3e728f8-c494-4293-8c8f-5dd77eda7630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:26.186074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-1786a223-af3c-4212-974f-f24a524e801d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86cb9f735ac14e56a948cf89fb6a43b9'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:26.187177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-1786a223-af3c-4212-974f-f24a524e801d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86cb9f735ac14e56a948cf89fb6a43b9'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:26.188073 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-1786a223-af3c-4212-974f-f24a524e801d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86cb9f735ac14e56a948cf89fb6a43b9'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:26.200542 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-1786a223-af3c-4212-974f-f24a524e801d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86cb9f735ac14e56a948cf89fb6a43b9'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:26.271714 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e728f8-c494-4293-8c8f-5dd77eda7630 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:26.277138 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cf265-fb3c-4364-975f-f8c4b9037a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:26.277969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cf265-fb3c-4364-975f-f8c4b9037a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:26.278254 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cf265-fb3c-4364-975f-f8c4b9037a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:26.278576 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cf265-fb3c-4364-975f-f8c4b9037a18 admin admin] PATH_INFO: `/v3/users/d94bccf167fa4d36903818e691d83775` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:26.279857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63cf265-fb3c-4364-975f-f8c4b9037a18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d94bccf167fa4d36903818e691d83775)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:26.282167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63cf265-fb3c-4364-975f-f8c4b9037a18 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:26.295913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:26.310147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-1786a223-af3c-4212-974f-f24a524e801d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86cb9f735ac14e56a948cf89fb6a43b9'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:26.329826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 968/1946] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:25 2022] DELETE /identity/v3/users/86cb9f735ac14e56a948cf89fb6a43b9 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:26.334130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15553c-e871-41b1-bf4b-b58702fec6b6 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:26.334425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15553c-e871-41b1-bf4b-b58702fec6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:26.334703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15553c-e871-41b1-bf4b-b58702fec6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:26.341631 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:26.354199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-ed15553c-e871-41b1-bf4b-b58702fec6b6 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:26.355446 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ed15553c-e871-41b1-bf4b-b58702fec6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:26.422704 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 969/1947] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:26.433710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15553c-e871-41b1-bf4b-b58702fec6b6 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:26.436520 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ed15553c-e871-41b1-bf4b-b58702fec6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:26.494206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15553c-e871-41b1-bf4b-b58702fec6b6 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:26.497234 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d63cf265-fb3c-4364-975f-f8c4b9037a18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd94bccf167fa4d36903818e691d83775'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:26.498019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d63cf265-fb3c-4364-975f-f8c4b9037a18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd94bccf167fa4d36903818e691d83775'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:26.498741 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9743c56-8466-44b7-90fd-e6d16ae315b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:26.499127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d63cf265-fb3c-4364-975f-f8c4b9037a18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd94bccf167fa4d36903818e691d83775'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:26.499357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9743c56-8466-44b7-90fd-e6d16ae315b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:26.499645 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9743c56-8466-44b7-90fd-e6d16ae315b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:26.499949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9743c56-8466-44b7-90fd-e6d16ae315b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:26.501782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b9743c56-8466-44b7-90fd-e6d16ae315b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:26.514031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9743c56-8466-44b7-90fd-e6d16ae315b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:26.515474 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9743c56-8466-44b7-90fd-e6d16ae315b6 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:26.525267 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 970/1948] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:49:26.592718 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d63cf265-fb3c-4364-975f-f8c4b9037a18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd94bccf167fa4d36903818e691d83775'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:26.600700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d63cf265-fb3c-4364-975f-f8c4b9037a18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd94bccf167fa4d36903818e691d83775'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:26.622401 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 979/1949] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:26 2022] DELETE /identity/v3/users/d94bccf167fa4d36903818e691d83775 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:27.455502 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9743c56-8466-44b7-90fd-e6d16ae315b6 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:27.457450 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cf265-fb3c-4364-975f-f8c4b9037a18 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:27.459012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b9743c56-8466-44b7-90fd-e6d16ae315b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:27.461554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d63cf265-fb3c-4364-975f-f8c4b9037a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:27.514096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9743c56-8466-44b7-90fd-e6d16ae315b6 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:27.653862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cf265-fb3c-4364-975f-f8c4b9037a18 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:27.660566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d7829-0574-4719-b125-e5309a730a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:27.662609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d7829-0574-4719-b125-e5309a730a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:27.663095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d7829-0574-4719-b125-e5309a730a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:27.663573 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d7829-0574-4719-b125-e5309a730a76 admin admin] PATH_INFO: `/v3/projects/fa2ae0e1d751435da1bdcacac5b21970` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:27.665019 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d7829-0574-4719-b125-e5309a730a76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa2ae0e1d751435da1bdcacac5b21970)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:27.667851 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d7829-0574-4719-b125-e5309a730a76 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:27.668074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f32f66-c024-40cf-9b23-1e08f766f89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:27.668795 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f32f66-c024-40cf-9b23-1e08f766f89d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:27.676825 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f32f66-c024-40cf-9b23-1e08f766f89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:27.677227 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f32f66-c024-40cf-9b23-1e08f766f89d admin admin] PATH_INFO: `/v3/projects/c2860711c8994a09b051d8806184fc12` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:27.678670 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f32f66-c024-40cf-9b23-1e08f766f89d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2860711c8994a09b051d8806184fc12)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:27.680826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f32f66-c024-40cf-9b23-1e08f766f89d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:27.804201 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-57f32f66-c024-40cf-9b23-1e08f766f89d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2860711c8994a09b051d8806184fc12', 'request_id': 'req-57f32f66-c024-40cf-9b23-1e08f766f89d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:27.808704 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-e26d7829-0574-4719-b125-e5309a730a76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa2ae0e1d751435da1bdcacac5b21970', 'request_id': 'req-e26d7829-0574-4719-b125-e5309a730a76'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:27.813803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-57f32f66-c024-40cf-9b23-1e08f766f89d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2860711c8994a09b051d8806184fc12', 'request_id': 'req-57f32f66-c024-40cf-9b23-1e08f766f89d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:27.818059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-e26d7829-0574-4719-b125-e5309a730a76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa2ae0e1d751435da1bdcacac5b21970', 'request_id': 'req-e26d7829-0574-4719-b125-e5309a730a76'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:27.820197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-57f32f66-c024-40cf-9b23-1e08f766f89d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2860711c8994a09b051d8806184fc12', 'request_id': 'req-57f32f66-c024-40cf-9b23-1e08f766f89d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:27.826944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-e26d7829-0574-4719-b125-e5309a730a76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa2ae0e1d751435da1bdcacac5b21970', 'request_id': 'req-e26d7829-0574-4719-b125-e5309a730a76'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:27.917649 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-57f32f66-c024-40cf-9b23-1e08f766f89d admin admin] The token cache is being invalidate because project c2860711c8994a09b051d8806184fc12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:27.918195 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-57f32f66-c024-40cf-9b23-1e08f766f89d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:27.919327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-57f32f66-c024-40cf-9b23-1e08f766f89d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:27.921508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 980/1950] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:27 2022] DELETE /identity/v3/projects/c2860711c8994a09b051d8806184fc12 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:27.927315 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087e18f-867e-4f5d-90d6-1bde5bc5e483 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:27.927686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087e18f-867e-4f5d-90d6-1bde5bc5e483 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:27.928041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087e18f-867e-4f5d-90d6-1bde5bc5e483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:27.948596 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a087e18f-867e-4f5d-90d6-1bde5bc5e483 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:27.949939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a087e18f-867e-4f5d-90d6-1bde5bc5e483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:27.964600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:27.967749 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:28.006855 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-e26d7829-0574-4719-b125-e5309a730a76 admin admin] The token cache is being invalidate because project fa2ae0e1d751435da1bdcacac5b21970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:28.007056 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-e26d7829-0574-4719-b125-e5309a730a76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:28.007681 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-e26d7829-0574-4719-b125-e5309a730a76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:28.009425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 971/1951] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:27 2022] DELETE /identity/v3/projects/fa2ae0e1d751435da1bdcacac5b21970 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:28.013701 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d7829-0574-4719-b125-e5309a730a76 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:28.015053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e26d7829-0574-4719-b125-e5309a730a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:28.017492 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 981/1952] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:28.027605 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115220f-eaba-4a62-8054-1d7fecece308 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:28.027909 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115220f-eaba-4a62-8054-1d7fecece308 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:28.028181 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115220f-eaba-4a62-8054-1d7fecece308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:28.028530 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d7829-0574-4719-b125-e5309a730a76 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:28.035074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af451fc-a7d2-44da-ad92-ad012a3cb205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:28.035437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af451fc-a7d2-44da-ad92-ad012a3cb205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:28.036089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af451fc-a7d2-44da-ad92-ad012a3cb205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:28.036373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af451fc-a7d2-44da-ad92-ad012a3cb205 admin admin] PATH_INFO: `/v3/users/d3b82f6b26214b7ab35d0e662dc97e96` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:28.037654 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af451fc-a7d2-44da-ad92-ad012a3cb205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3b82f6b26214b7ab35d0e662dc97e96)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:28.039594 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af451fc-a7d2-44da-ad92-ad012a3cb205 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:28.050551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8115220f-eaba-4a62-8054-1d7fecece308 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:28.051908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8115220f-eaba-4a62-8054-1d7fecece308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:28.063984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 982/1953] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:28.069814 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115220f-eaba-4a62-8054-1d7fecece308 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:28.075177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115220f-eaba-4a62-8054-1d7fecece308 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:28.079843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb688d-bd2a-462a-ad19-be43a39391b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:28.080491 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb688d-bd2a-462a-ad19-be43a39391b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:28.080790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb688d-bd2a-462a-ad19-be43a39391b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:28.081030 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb688d-bd2a-462a-ad19-be43a39391b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:28.081670 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb688d-bd2a-462a-ad19-be43a39391b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:49:28.081971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb688d-bd2a-462a-ad19-be43a39391b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:28.083579 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb688d-bd2a-462a-ad19-be43a39391b0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:28.100169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 983/1954] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:49:28.101308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb688d-bd2a-462a-ad19-be43a39391b0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:28.111438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb688d-bd2a-462a-ad19-be43a39391b0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:28.116391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df458e-5aa4-47aa-9cb6-a3ebd5e38539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:28.117018 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df458e-5aa4-47aa-9cb6-a3ebd5e38539 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:28.117305 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df458e-5aa4-47aa-9cb6-a3ebd5e38539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:28.118399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df458e-5aa4-47aa-9cb6-a3ebd5e38539 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:28.118676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df458e-5aa4-47aa-9cb6-a3ebd5e38539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:28.121055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df458e-5aa4-47aa-9cb6-a3ebd5e38539 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:28.307308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 984/1955] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:28 2022] POST /identity/v3/projects => generated 367 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:28.311248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df458e-5aa4-47aa-9cb6-a3ebd5e38539 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:28.327079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-5af451fc-a7d2-44da-ad92-ad012a3cb205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3b82f6b26214b7ab35d0e662dc97e96'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:28.327876 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-5af451fc-a7d2-44da-ad92-ad012a3cb205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3b82f6b26214b7ab35d0e662dc97e96'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:28.330343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-5af451fc-a7d2-44da-ad92-ad012a3cb205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3b82f6b26214b7ab35d0e662dc97e96'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:28.352560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-5af451fc-a7d2-44da-ad92-ad012a3cb205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3b82f6b26214b7ab35d0e662dc97e96'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:28.432632 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df458e-5aa4-47aa-9cb6-a3ebd5e38539 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:28.437688 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291bb46-a16e-491e-aef9-eb7ff88faa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:28.438305 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291bb46-a16e-491e-aef9-eb7ff88faa9e admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:28.438610 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291bb46-a16e-491e-aef9-eb7ff88faa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:28.438871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291bb46-a16e-491e-aef9-eb7ff88faa9e admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:28.439553 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291bb46-a16e-491e-aef9-eb7ff88faa9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:28.441638 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291bb46-a16e-491e-aef9-eb7ff88faa9e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:28.521173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-5af451fc-a7d2-44da-ad92-ad012a3cb205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3b82f6b26214b7ab35d0e662dc97e96'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:28.551245 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 985/1956] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:28 2022] POST /identity/v3/users => generated 284 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:28.558330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291bb46-a16e-491e-aef9-eb7ff88faa9e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:28.560133 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c291bb46-a16e-491e-aef9-eb7ff88faa9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:28.563614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 972/1957] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:28 2022] DELETE /identity/v3/users/d3b82f6b26214b7ab35d0e662dc97e96 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:28.574687 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291bb46-a16e-491e-aef9-eb7ff88faa9e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:28.657652 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345a8c3-9835-448b-9f0e-65f21fac7ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:28.658404 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345a8c3-9835-448b-9f0e-65f21fac7ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:28.658692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345a8c3-9835-448b-9f0e-65f21fac7ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:28.658960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345a8c3-9835-448b-9f0e-65f21fac7ce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:28.659635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345a8c3-9835-448b-9f0e-65f21fac7ce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:28.661361 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345a8c3-9835-448b-9f0e-65f21fac7ce7 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:28.677978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 986/1958] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:49:28.681656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af451fc-a7d2-44da-ad92-ad012a3cb205 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:28.690067 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af451fc-a7d2-44da-ad92-ad012a3cb205 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:28.694888 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c055b-33f7-4547-855d-07ebac6f7bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:28.695499 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c055b-33f7-4547-855d-07ebac6f7bcd admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:28.695781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c055b-33f7-4547-855d-07ebac6f7bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:28.696054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c055b-33f7-4547-855d-07ebac6f7bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:28.696726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c055b-33f7-4547-855d-07ebac6f7bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:28.698374 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c055b-33f7-4547-855d-07ebac6f7bcd admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:28.708980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 973/1959] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:28.713652 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345a8c3-9835-448b-9f0e-65f21fac7ce7 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:28.723853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345a8c3-9835-448b-9f0e-65f21fac7ce7 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:28.729711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8a3fe-7b54-4520-b023-a547d44a7e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:28.730384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8a3fe-7b54-4520-b023-a547d44a7e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:28.730663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8a3fe-7b54-4520-b023-a547d44a7e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:28.732529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8a3fe-7b54-4520-b023-a547d44a7e17 admin admin] PATH_INFO: `/v3/projects/5002b5e821e84c2fa6de04d96a318f7b/users/398b57c208c642f6bcbe551c1891e42a/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:28.747594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8a3fe-7b54-4520-b023-a547d44a7e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5002b5e821e84c2fa6de04d96a318f7b, user_id=398b57c208c642f6bcbe551c1891e42a, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:28.750498 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8a3fe-7b54-4520-b023-a547d44a7e17 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:28.854278 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 987/1960] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:28 2022] PUT /identity/v3/projects/5002b5e821e84c2fa6de04d96a318f7b/users/398b57c208c642f6bcbe551c1891e42a/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:28.860355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87bd82-fdeb-4de1-b5a1-17e6f68c889b None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:28.860897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87bd82-fdeb-4de1-b5a1-17e6f68c889b None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:28.861296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87bd82-fdeb-4de1-b5a1-17e6f68c889b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:28.896329 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-cc87bd82-fdeb-4de1-b5a1-17e6f68c889b None None] MFA Rules not processed for user `398b57c208c642f6bcbe551c1891e42a`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:28.897828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-cc87bd82-fdeb-4de1-b5a1-17e6f68c889b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:28.969124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 974/1961] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:28.981448 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8a3fe-7b54-4520-b023-a547d44a7e17 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:28.983250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-42e8a3fe-7b54-4520-b023-a547d44a7e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:29.065015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8a3fe-7b54-4520-b023-a547d44a7e17 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:29.068666 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050add8-d942-44ce-ab6e-426ea93dcfe0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56418ff737504181a59714aad9f1c375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:29.069311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050add8-d942-44ce-ab6e-426ea93dcfe0 service glance] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:29.069591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050add8-d942-44ce-ab6e-426ea93dcfe0 service glance] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:29.069864 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050add8-d942-44ce-ab6e-426ea93dcfe0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:29.080148 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050add8-d942-44ce-ab6e-426ea93dcfe0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:29.082388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050add8-d942-44ce-ab6e-426ea93dcfe0 service glance] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:29.094057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 988/1962] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:29.139122 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:29.311615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87bd82-fdeb-4de1-b5a1-17e6f68c889b None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:29.313076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-cc87bd82-fdeb-4de1-b5a1-17e6f68c889b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:29.362375 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:29.402504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87bd82-fdeb-4de1-b5a1-17e6f68c889b None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:29.407666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e72ed-101b-443e-ba14-5f46fbe69284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:29.408629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e72ed-101b-443e-ba14-5f46fbe69284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:29.409087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e72ed-101b-443e-ba14-5f46fbe69284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:29.409560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e72ed-101b-443e-ba14-5f46fbe69284 admin admin] PATH_INFO: `/v3/projects/a825738e136c404e841788bf92daa81c` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:29.410909 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e72ed-101b-443e-ba14-5f46fbe69284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a825738e136c404e841788bf92daa81c)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:29.413058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e72ed-101b-443e-ba14-5f46fbe69284 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:29.536109 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-f60e72ed-101b-443e-ba14-5f46fbe69284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a825738e136c404e841788bf92daa81c', 'request_id': 'req-f60e72ed-101b-443e-ba14-5f46fbe69284'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:29.548980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-f60e72ed-101b-443e-ba14-5f46fbe69284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a825738e136c404e841788bf92daa81c', 'request_id': 'req-f60e72ed-101b-443e-ba14-5f46fbe69284'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:29.557446 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-f60e72ed-101b-443e-ba14-5f46fbe69284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a825738e136c404e841788bf92daa81c', 'request_id': 'req-f60e72ed-101b-443e-ba14-5f46fbe69284'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:29.715682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-f60e72ed-101b-443e-ba14-5f46fbe69284 admin admin] The token cache is being invalidate because project a825738e136c404e841788bf92daa81c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:29.716123 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-f60e72ed-101b-443e-ba14-5f46fbe69284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:29.716885 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-f60e72ed-101b-443e-ba14-5f46fbe69284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:29.718952 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 975/1963] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:29 2022] DELETE /identity/v3/projects/a825738e136c404e841788bf92daa81c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:29.723594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0571b-64bc-4558-8840-65ad0ec54fce None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:29.724638 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0571b-64bc-4558-8840-65ad0ec54fce None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:29.725066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0571b-64bc-4558-8840-65ad0ec54fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:29.748877 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-80a0571b-64bc-4558-8840-65ad0ec54fce None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:29.750110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-80a0571b-64bc-4558-8840-65ad0ec54fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:29.818779 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 989/1964] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:29.828143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08159ea4-6157-40c9-a722-8837e6380ca4 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:29.829395 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08159ea4-6157-40c9-a722-8837e6380ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:29.829853 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08159ea4-6157-40c9-a722-8837e6380ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:29.859688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-08159ea4-6157-40c9-a722-8837e6380ca4 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:29.860181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-08159ea4-6157-40c9-a722-8837e6380ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:29.870504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 976/1965] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:29.875195 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0571b-64bc-4558-8840-65ad0ec54fce None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:29.888121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0571b-64bc-4558-8840-65ad0ec54fce None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:29.897146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934889ae-f643-4623-9f00-441cfe986c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:29.897781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934889ae-f643-4623-9f00-441cfe986c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:29.897781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934889ae-f643-4623-9f00-441cfe986c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:29.897979 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934889ae-f643-4623-9f00-441cfe986c2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:29.898480 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934889ae-f643-4623-9f00-441cfe986c2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:49:29.898786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934889ae-f643-4623-9f00-441cfe986c2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:29.901055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934889ae-f643-4623-9f00-441cfe986c2b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:29.920158 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 990/1966] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:49:29.925799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08159ea4-6157-40c9-a722-8837e6380ca4 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:29.936488 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08159ea4-6157-40c9-a722-8837e6380ca4 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:29.941349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ea605-97b5-4ffb-a1e8-f6202b949854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:29.941973 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ea605-97b5-4ffb-a1e8-f6202b949854 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:29.942295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ea605-97b5-4ffb-a1e8-f6202b949854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:29.942784 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ea605-97b5-4ffb-a1e8-f6202b949854 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:29.943472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ea605-97b5-4ffb-a1e8-f6202b949854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:29.945812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ea605-97b5-4ffb-a1e8-f6202b949854 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:30.037402 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 977/1967] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:29 2022] POST /identity/v3/projects => generated 363 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:30.041217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934889ae-f643-4623-9f00-441cfe986c2b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:30.059444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934889ae-f643-4623-9f00-441cfe986c2b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:30.176503 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:30.193516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1aaf9-5322-4497-8f82-b5ce2edecd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:30.194298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1aaf9-5322-4497-8f82-b5ce2edecd50 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:30.194779 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1aaf9-5322-4497-8f82-b5ce2edecd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:30.195262 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1aaf9-5322-4497-8f82-b5ce2edecd50 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:30.196441 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1aaf9-5322-4497-8f82-b5ce2edecd50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:30.199619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1aaf9-5322-4497-8f82-b5ce2edecd50 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:30.311482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 991/1968] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:30 2022] POST /identity/v3/users => generated 282 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:30.312700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ea605-97b5-4ffb-a1e8-f6202b949854 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:30.324430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ea605-97b5-4ffb-a1e8-f6202b949854 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:30.329097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77042d9-bc2b-47e0-abbd-f76aa653ffcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:30.329704 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77042d9-bc2b-47e0-abbd-f76aa653ffcb admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:30.330035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77042d9-bc2b-47e0-abbd-f76aa653ffcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:30.330330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77042d9-bc2b-47e0-abbd-f76aa653ffcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:30.330968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77042d9-bc2b-47e0-abbd-f76aa653ffcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:30.332586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77042d9-bc2b-47e0-abbd-f76aa653ffcb admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:30.347021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 978/1969] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:30.349619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1aaf9-5322-4497-8f82-b5ce2edecd50 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:30.361493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1aaf9-5322-4497-8f82-b5ce2edecd50 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:30.366540 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bb0ca-8dd5-4dee-98a5-c9dda0416b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:30.367284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bb0ca-8dd5-4dee-98a5-c9dda0416b10 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:30.377793 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bb0ca-8dd5-4dee-98a5-c9dda0416b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:30.377793 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bb0ca-8dd5-4dee-98a5-c9dda0416b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:30.377793 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bb0ca-8dd5-4dee-98a5-c9dda0416b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:30.377793 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bb0ca-8dd5-4dee-98a5-c9dda0416b10 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:30.382459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 992/1970] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:30.387778 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77042d9-bc2b-47e0-abbd-f76aa653ffcb admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:30.415115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77042d9-bc2b-47e0-abbd-f76aa653ffcb admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:30.420821 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25aa51d-d2ad-417f-9f7d-9fbc8d88e34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:30.421597 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25aa51d-d2ad-417f-9f7d-9fbc8d88e34c admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:30.421886 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25aa51d-d2ad-417f-9f7d-9fbc8d88e34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:30.422161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25aa51d-d2ad-417f-9f7d-9fbc8d88e34c admin admin] PATH_INFO: `/v3/projects/d1648b4e04654f059e9a7486c0092122/users/f4c7e88187f14d0aba6c3c7c9d339905/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:30.438190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25aa51d-d2ad-417f-9f7d-9fbc8d88e34c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1648b4e04654f059e9a7486c0092122, user_id=f4c7e88187f14d0aba6c3c7c9d339905, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:30.441257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25aa51d-d2ad-417f-9f7d-9fbc8d88e34c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:30.527682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 979/1971] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:30 2022] PUT /identity/v3/projects/d1648b4e04654f059e9a7486c0092122/users/f4c7e88187f14d0aba6c3c7c9d339905/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:30.539579 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dcaf1-9cc5-4409-903d-29fa71ddebe0 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:30.539866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dcaf1-9cc5-4409-903d-29fa71ddebe0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:30.540146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dcaf1-9cc5-4409-903d-29fa71ddebe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:30.576065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-337dcaf1-9cc5-4409-903d-29fa71ddebe0 None None] MFA Rules not processed for user `f4c7e88187f14d0aba6c3c7c9d339905`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:30.577582 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-337dcaf1-9cc5-4409-903d-29fa71ddebe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:30.650806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 993/1972] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:30.655363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25aa51d-d2ad-417f-9f7d-9fbc8d88e34c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:30.722110 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:30.733197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25aa51d-d2ad-417f-9f7d-9fbc8d88e34c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:30.738332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3980956-8f87-4bf3-8414-4efa743997f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:30.739102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3980956-8f87-4bf3-8414-4efa743997f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:30.739443 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3980956-8f87-4bf3-8414-4efa743997f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:30.739828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3980956-8f87-4bf3-8414-4efa743997f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:30.741436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3980956-8f87-4bf3-8414-4efa743997f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:30.742534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3980956-8f87-4bf3-8414-4efa743997f8 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:30.867027 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 980/1973] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:30 2022] POST /identity/v3/projects => generated 363 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:30.868824 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dcaf1-9cc5-4409-903d-29fa71ddebe0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:30.886970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dcaf1-9cc5-4409-903d-29fa71ddebe0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:30.995641 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afd3b8-745b-415a-b0f6-fa6091b195a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:30.996249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afd3b8-745b-415a-b0f6-fa6091b195a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:30.996563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afd3b8-745b-415a-b0f6-fa6091b195a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:30.996838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afd3b8-745b-415a-b0f6-fa6091b195a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:30.997460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3afd3b8-745b-415a-b0f6-fa6091b195a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:30.999365 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3afd3b8-745b-415a-b0f6-fa6091b195a3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:31.138443 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 994/1974] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:30 2022] POST /identity/v3/users => generated 288 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:31.139753 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3980956-8f87-4bf3-8414-4efa743997f8 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:31.156247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3980956-8f87-4bf3-8414-4efa743997f8 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:31.170956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e3c7c-d209-4429-aeb5-33f1dca307e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:31.172989 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e3c7c-d209-4429-aeb5-33f1dca307e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:31.173810 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e3c7c-d209-4429-aeb5-33f1dca307e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:31.174720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e3c7c-d209-4429-aeb5-33f1dca307e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:31.176202 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e3c7c-d209-4429-aeb5-33f1dca307e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:31.179841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e3c7c-d209-4429-aeb5-33f1dca307e4 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:31.196192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 981/1975] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:31.201258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afd3b8-745b-415a-b0f6-fa6091b195a3 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:31.218317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afd3b8-745b-415a-b0f6-fa6091b195a3 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:31.226079 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bd0ac-4b69-4c3d-afca-8a4f5c8433a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:31.230975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bd0ac-4b69-4c3d-afca-8a4f5c8433a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:31.230975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bd0ac-4b69-4c3d-afca-8a4f5c8433a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:31.230975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bd0ac-4b69-4c3d-afca-8a4f5c8433a9 admin admin] PATH_INFO: `/v3/projects/d23928ad23834d199ddf10e3e445771f/users/1c6f10c637d745ffbd756e45014952bd/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:31.248679 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7bd0ac-4b69-4c3d-afca-8a4f5c8433a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23928ad23834d199ddf10e3e445771f, user_id=1c6f10c637d745ffbd756e45014952bd, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:31.251552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7bd0ac-4b69-4c3d-afca-8a4f5c8433a9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:31.365342 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 995/1976] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:31 2022] PUT /identity/v3/projects/d23928ad23834d199ddf10e3e445771f/users/1c6f10c637d745ffbd756e45014952bd/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:31.370428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e3c7c-d209-4429-aeb5-33f1dca307e4 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:31.381207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e3c7c-d209-4429-aeb5-33f1dca307e4 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:31.454243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32917aab-af11-4385-a339-6c4485dd412c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:31.455425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32917aab-af11-4385-a339-6c4485dd412c admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:31.455973 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32917aab-af11-4385-a339-6c4485dd412c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:31.456597 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32917aab-af11-4385-a339-6c4485dd412c admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:31.457703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32917aab-af11-4385-a339-6c4485dd412c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:31.459964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32917aab-af11-4385-a339-6c4485dd412c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:31.472805 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 982/1977] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:31.476864 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bd0ac-4b69-4c3d-afca-8a4f5c8433a9 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:31.488836 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bd0ac-4b69-4c3d-afca-8a4f5c8433a9 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:31.491949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32917aab-af11-4385-a339-6c4485dd412c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:31.494260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-32917aab-af11-4385-a339-6c4485dd412c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:31.495075 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd37bc-1e42-4426-af36-0e6452c67bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:31.495877 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd37bc-1e42-4426-af36-0e6452c67bbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:31.496364 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd37bc-1e42-4426-af36-0e6452c67bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:31.496773 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd37bc-1e42-4426-af36-0e6452c67bbf admin admin] PATH_INFO: `/v3/projects/d23928ad23834d199ddf10e3e445771f/users/1c6f10c637d745ffbd756e45014952bd/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:31.499452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd37bc-1e42-4426-af36-0e6452c67bbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23928ad23834d199ddf10e3e445771f, user_id=1c6f10c637d745ffbd756e45014952bd, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:31.502280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd37bc-1e42-4426-af36-0e6452c67bbf admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:31.508465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32917aab-af11-4385-a339-6c4485dd412c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:31.514356 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266223a-3778-4089-b92c-42fd383c795c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:31.515014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266223a-3778-4089-b92c-42fd383c795c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:31.515360 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266223a-3778-4089-b92c-42fd383c795c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:31.515791 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266223a-3778-4089-b92c-42fd383c795c admin admin] PATH_INFO: `/v3/users/1715248c7cd84dc982e8e56d726edb2c` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:31.517101 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266223a-3778-4089-b92c-42fd383c795c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1715248c7cd84dc982e8e56d726edb2c)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:31.519283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266223a-3778-4089-b92c-42fd383c795c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:31.520487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 996/1978] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:31 2022] PUT /identity/v3/projects/d23928ad23834d199ddf10e3e445771f/users/1c6f10c637d745ffbd756e45014952bd/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:31.529325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b69d8-1239-4c10-9400-2a819f41d896 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:31.529806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b69d8-1239-4c10-9400-2a819f41d896 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:31.530063 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b69d8-1239-4c10-9400-2a819f41d896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:31.554716 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:31.564130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e75b69d8-1239-4c10-9400-2a819f41d896 None None] MFA Rules not processed for user `1c6f10c637d745ffbd756e45014952bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:31.567333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e75b69d8-1239-4c10-9400-2a819f41d896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:31.641679 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 997/1979] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:31.654786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b69d8-1239-4c10-9400-2a819f41d896 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:31.656462 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e75b69d8-1239-4c10-9400-2a819f41d896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:31.725939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b69d8-1239-4c10-9400-2a819f41d896 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:31.730122 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d8141-7d96-41b7-9709-d97f792f67cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:31.730738 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d8141-7d96-41b7-9709-d97f792f67cf service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:31.731024 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d8141-7d96-41b7-9709-d97f792f67cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:31.731448 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d8141-7d96-41b7-9709-d97f792f67cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:31.741087 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d8141-7d96-41b7-9709-d97f792f67cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:31.743707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d8141-7d96-41b7-9709-d97f792f67cf service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:31.752139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7266223a-3778-4089-b92c-42fd383c795c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1715248c7cd84dc982e8e56d726edb2c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:31.752872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7266223a-3778-4089-b92c-42fd383c795c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1715248c7cd84dc982e8e56d726edb2c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:31.753495 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7266223a-3778-4089-b92c-42fd383c795c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1715248c7cd84dc982e8e56d726edb2c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:31.754476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 998/1980] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:49:31.762656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7266223a-3778-4089-b92c-42fd383c795c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1715248c7cd84dc982e8e56d726edb2c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:31.824073 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7266223a-3778-4089-b92c-42fd383c795c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1715248c7cd84dc982e8e56d726edb2c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:31.843392 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 983/1981] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:31 2022] DELETE /identity/v3/users/1715248c7cd84dc982e8e56d726edb2c => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:31.848016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6535b5-1499-4dda-a656-4078aa07ed66 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:31.848311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6535b5-1499-4dda-a656-4078aa07ed66 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:31.848642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6535b5-1499-4dda-a656-4078aa07ed66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:31.868896 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7b6535b5-1499-4dda-a656-4078aa07ed66 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:31.870682 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7b6535b5-1499-4dda-a656-4078aa07ed66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:31.974290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 999/1982] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:31.980844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266223a-3778-4089-b92c-42fd383c795c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:31.983341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7266223a-3778-4089-b92c-42fd383c795c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:32.044097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:32.075155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266223a-3778-4089-b92c-42fd383c795c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:32.079630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0931237-181b-442b-8672-30ea37fa538d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:32.080994 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0931237-181b-442b-8672-30ea37fa538d service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:32.092428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0931237-181b-442b-8672-30ea37fa538d service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:32.092428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0931237-181b-442b-8672-30ea37fa538d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:32.110090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0931237-181b-442b-8672-30ea37fa538d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:32.112506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0931237-181b-442b-8672-30ea37fa538d service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:32.126443 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 984/1983] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:32.615367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6535b5-1499-4dda-a656-4078aa07ed66 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:32.617179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7b6535b5-1499-4dda-a656-4078aa07ed66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:32.636186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6535b5-1499-4dda-a656-4078aa07ed66 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:32.641308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:32.642368 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:32.642368 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:32.642588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde admin admin] PATH_INFO: `/v3/projects/319820259eeb4b56b01b7698570ceff7` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:32.643726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=319820259eeb4b56b01b7698570ceff7)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:32.645684 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:32.784029 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:32.804918 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '319820259eeb4b56b01b7698570ceff7', 'request_id': 'req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:32.811512 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '319820259eeb4b56b01b7698570ceff7', 'request_id': 'req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:32.817010 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '319820259eeb4b56b01b7698570ceff7', 'request_id': 'req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:32.876112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0931237-181b-442b-8672-30ea37fa538d service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:32.877620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d0931237-181b-442b-8672-30ea37fa538d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:32.891184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0931237-181b-442b-8672-30ea37fa538d service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:32.907350 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde admin admin] The token cache is being invalidate because project 319820259eeb4b56b01b7698570ceff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:32.908124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:32.909849 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:32.914984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1000/1984] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:32 2022] DELETE /identity/v3/projects/319820259eeb4b56b01b7698570ceff7 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:32.916155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:32.918293 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:32.980430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf0674-1aa8-4f90-8058-b8e0f0025d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:32.981010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf0674-1aa8-4f90-8058-b8e0f0025d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:32.981299 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf0674-1aa8-4f90-8058-b8e0f0025d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:32.981579 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf0674-1aa8-4f90-8058-b8e0f0025d62 admin admin] PATH_INFO: `/v3/users/f4c7e88187f14d0aba6c3c7c9d339905` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:32.982719 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf0674-1aa8-4f90-8058-b8e0f0025d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4c7e88187f14d0aba6c3c7c9d339905)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:32.984688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf0674-1aa8-4f90-8058-b8e0f0025d62 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:33.004995 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6c988-f0d1-4e2a-87b8-b9d4ffb63cde admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:33.010174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83cf9c-96c6-4e74-9862-9880903924ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:33.010814 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83cf9c-96c6-4e74-9862-9880903924ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:33.011093 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83cf9c-96c6-4e74-9862-9880903924ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:33.011373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83cf9c-96c6-4e74-9862-9880903924ee admin admin] PATH_INFO: `/v3/users/33bd36b761a44df99f46e4fea9d991d3` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:33.013179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83cf9c-96c6-4e74-9862-9880903924ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=33bd36b761a44df99f46e4fea9d991d3)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:33.015250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83cf9c-96c6-4e74-9862-9880903924ee admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:33.231522 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a5bf0674-1aa8-4f90-8058-b8e0f0025d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4c7e88187f14d0aba6c3c7c9d339905'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:33.232082 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a5bf0674-1aa8-4f90-8058-b8e0f0025d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4c7e88187f14d0aba6c3c7c9d339905'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:33.232788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a5bf0674-1aa8-4f90-8058-b8e0f0025d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4c7e88187f14d0aba6c3c7c9d339905'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:33.241484 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a5bf0674-1aa8-4f90-8058-b8e0f0025d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4c7e88187f14d0aba6c3c7c9d339905'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:33.255660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8c83cf9c-96c6-4e74-9862-9880903924ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33bd36b761a44df99f46e4fea9d991d3'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:33.256323 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8c83cf9c-96c6-4e74-9862-9880903924ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33bd36b761a44df99f46e4fea9d991d3'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:33.256991 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8c83cf9c-96c6-4e74-9862-9880903924ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33bd36b761a44df99f46e4fea9d991d3'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:33.330015 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a5bf0674-1aa8-4f90-8058-b8e0f0025d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4c7e88187f14d0aba6c3c7c9d339905'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:33.352550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 985/1985] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:32 2022] DELETE /identity/v3/users/f4c7e88187f14d0aba6c3c7c9d339905 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:33.356586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278fa4f-6b21-4a35-9b6e-ffe8ce167294 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:33.356885 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278fa4f-6b21-4a35-9b6e-ffe8ce167294 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:33.357163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278fa4f-6b21-4a35-9b6e-ffe8ce167294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:33.377917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-9278fa4f-6b21-4a35-9b6e-ffe8ce167294 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:33.379112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-9278fa4f-6b21-4a35-9b6e-ffe8ce167294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:33.427421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8c83cf9c-96c6-4e74-9862-9880903924ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33bd36b761a44df99f46e4fea9d991d3'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:33.435538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8c83cf9c-96c6-4e74-9862-9880903924ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33bd36b761a44df99f46e4fea9d991d3'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:33.455027 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:33.462945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1001/1986] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:32 2022] DELETE /identity/v3/users/33bd36b761a44df99f46e4fea9d991d3 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:33.470284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 986/1987] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:33.491945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83cf9c-96c6-4e74-9862-9880903924ee admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:33.495019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8c83cf9c-96c6-4e74-9862-9880903924ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:33.563625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:33.575953 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83cf9c-96c6-4e74-9862-9880903924ee admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:33.579805 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62555347-5ee0-4544-8737-cf1e331d8d89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:33.580582 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62555347-5ee0-4544-8737-cf1e331d8d89 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:33.580868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62555347-5ee0-4544-8737-cf1e331d8d89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:33.581141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62555347-5ee0-4544-8737-cf1e331d8d89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:33.651578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62555347-5ee0-4544-8737-cf1e331d8d89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:33.652942 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62555347-5ee0-4544-8737-cf1e331d8d89 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:33.662944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1002/1988] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:34.345721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278fa4f-6b21-4a35-9b6e-ffe8ce167294 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:34.347345 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-9278fa4f-6b21-4a35-9b6e-ffe8ce167294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:34.364369 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278fa4f-6b21-4a35-9b6e-ffe8ce167294 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:34.373341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae44e01-57ea-4848-9321-06c21c304410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:34.374239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae44e01-57ea-4848-9321-06c21c304410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:34.374682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae44e01-57ea-4848-9321-06c21c304410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:34.375140 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae44e01-57ea-4848-9321-06c21c304410 admin admin] PATH_INFO: `/v3/projects/5de0589689b4485e9168630e90f5f2b3` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:34.376878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae44e01-57ea-4848-9321-06c21c304410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5de0589689b4485e9168630e90f5f2b3)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:34.380000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae44e01-57ea-4848-9321-06c21c304410 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:34.519122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fae44e01-57ea-4848-9321-06c21c304410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5de0589689b4485e9168630e90f5f2b3', 'request_id': 'req-fae44e01-57ea-4848-9321-06c21c304410'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:34.527256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fae44e01-57ea-4848-9321-06c21c304410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5de0589689b4485e9168630e90f5f2b3', 'request_id': 'req-fae44e01-57ea-4848-9321-06c21c304410'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:34.536360 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fae44e01-57ea-4848-9321-06c21c304410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5de0589689b4485e9168630e90f5f2b3', 'request_id': 'req-fae44e01-57ea-4848-9321-06c21c304410'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:34.620989 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fae44e01-57ea-4848-9321-06c21c304410 admin admin] The token cache is being invalidate because project 5de0589689b4485e9168630e90f5f2b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:34.621337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fae44e01-57ea-4848-9321-06c21c304410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:34.622297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fae44e01-57ea-4848-9321-06c21c304410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:34.624257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 987/1989] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:34 2022] DELETE /identity/v3/projects/5de0589689b4485e9168630e90f5f2b3 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:34.628243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62555347-5ee0-4544-8737-cf1e331d8d89 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:34.639739 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62555347-5ee0-4544-8737-cf1e331d8d89 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:34.711031 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476295f3-5d54-488b-9475-b2b81420746f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:34.711700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476295f3-5d54-488b-9475-b2b81420746f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:34.711985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476295f3-5d54-488b-9475-b2b81420746f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:34.712261 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476295f3-5d54-488b-9475-b2b81420746f admin admin] PATH_INFO: `/v3/users/4e69211197d1412caa2799b16676fef6` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:34.713498 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476295f3-5d54-488b-9475-b2b81420746f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e69211197d1412caa2799b16676fef6)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:34.716801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476295f3-5d54-488b-9475-b2b81420746f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:34.841285 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae44e01-57ea-4848-9321-06c21c304410 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:34.844563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-fae44e01-57ea-4848-9321-06c21c304410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:34.863606 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae44e01-57ea-4848-9321-06c21c304410 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:34.871202 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a03778-6f09-4bbb-a47e-40d1707fcf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:34.871202 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a03778-6f09-4bbb-a47e-40d1707fcf42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:34.871202 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a03778-6f09-4bbb-a47e-40d1707fcf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:34.871754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a03778-6f09-4bbb-a47e-40d1707fcf42 admin admin] PATH_INFO: `/v3/projects/d1648b4e04654f059e9a7486c0092122` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:34.872681 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a03778-6f09-4bbb-a47e-40d1707fcf42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1648b4e04654f059e9a7486c0092122)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:34.874643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a03778-6f09-4bbb-a47e-40d1707fcf42 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:34.908463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-476295f3-5d54-488b-9475-b2b81420746f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e69211197d1412caa2799b16676fef6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:34.909214 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-476295f3-5d54-488b-9475-b2b81420746f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e69211197d1412caa2799b16676fef6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:34.909791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-476295f3-5d54-488b-9475-b2b81420746f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e69211197d1412caa2799b16676fef6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.010777 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-476295f3-5d54-488b-9475-b2b81420746f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e69211197d1412caa2799b16676fef6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.025470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-476295f3-5d54-488b-9475-b2b81420746f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e69211197d1412caa2799b16676fef6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.026650 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-45a03778-6f09-4bbb-a47e-40d1707fcf42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1648b4e04654f059e9a7486c0092122', 'request_id': 'req-45a03778-6f09-4bbb-a47e-40d1707fcf42'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.035603 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-45a03778-6f09-4bbb-a47e-40d1707fcf42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1648b4e04654f059e9a7486c0092122', 'request_id': 'req-45a03778-6f09-4bbb-a47e-40d1707fcf42'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.045300 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-45a03778-6f09-4bbb-a47e-40d1707fcf42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1648b4e04654f059e9a7486c0092122', 'request_id': 'req-45a03778-6f09-4bbb-a47e-40d1707fcf42'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.054185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1003/1990] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:34 2022] DELETE /identity/v3/users/4e69211197d1412caa2799b16676fef6 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:35.124066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-45a03778-6f09-4bbb-a47e-40d1707fcf42 admin admin] The token cache is being invalidate because project d1648b4e04654f059e9a7486c0092122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:35.125077 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-45a03778-6f09-4bbb-a47e-40d1707fcf42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.125579 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-45a03778-6f09-4bbb-a47e-40d1707fcf42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.128422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 988/1991] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:34 2022] DELETE /identity/v3/projects/d1648b4e04654f059e9a7486c0092122 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:35.132301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476295f3-5d54-488b-9475-b2b81420746f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:35.134381 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-476295f3-5d54-488b-9475-b2b81420746f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:35.148576 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:35.209224 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476295f3-5d54-488b-9475-b2b81420746f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:35.216005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdce6f9-4c3f-4e28-8f94-59f64285ae3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:35.216798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdce6f9-4c3f-4e28-8f94-59f64285ae3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:35.217152 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdce6f9-4c3f-4e28-8f94-59f64285ae3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:35.217539 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdce6f9-4c3f-4e28-8f94-59f64285ae3f admin admin] PATH_INFO: `/v3/users/1c6f10c637d745ffbd756e45014952bd` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:35.218918 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdce6f9-4c3f-4e28-8f94-59f64285ae3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c6f10c637d745ffbd756e45014952bd)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:35.220965 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdce6f9-4c3f-4e28-8f94-59f64285ae3f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:35.429341 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-1cdce6f9-4c3f-4e28-8f94-59f64285ae3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c6f10c637d745ffbd756e45014952bd'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.430017 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-1cdce6f9-4c3f-4e28-8f94-59f64285ae3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c6f10c637d745ffbd756e45014952bd'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.431013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-1cdce6f9-4c3f-4e28-8f94-59f64285ae3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c6f10c637d745ffbd756e45014952bd'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.533607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-1cdce6f9-4c3f-4e28-8f94-59f64285ae3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c6f10c637d745ffbd756e45014952bd'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.550129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a03778-6f09-4bbb-a47e-40d1707fcf42 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:35.552321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-45a03778-6f09-4bbb-a47e-40d1707fcf42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:35.563847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-1cdce6f9-4c3f-4e28-8f94-59f64285ae3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c6f10c637d745ffbd756e45014952bd'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.591212 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1004/1992] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:35 2022] DELETE /identity/v3/users/1c6f10c637d745ffbd756e45014952bd => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:35.652323 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a03778-6f09-4bbb-a47e-40d1707fcf42 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:35.661393 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:35.716108 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3844511-a0c3-4214-afe0-ce6da5483ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:35.716927 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3844511-a0c3-4214-afe0-ce6da5483ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:35.717213 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3844511-a0c3-4214-afe0-ce6da5483ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:35.717454 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3844511-a0c3-4214-afe0-ce6da5483ec5 admin admin] PATH_INFO: `/v3/projects/56f05d774edb4917bb4926dc03516d69` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:35.718741 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3844511-a0c3-4214-afe0-ce6da5483ec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f05d774edb4917bb4926dc03516d69)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:35.720711 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3844511-a0c3-4214-afe0-ce6da5483ec5 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:35.834954 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d3844511-a0c3-4214-afe0-ce6da5483ec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f05d774edb4917bb4926dc03516d69', 'request_id': 'req-d3844511-a0c3-4214-afe0-ce6da5483ec5'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.842290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d3844511-a0c3-4214-afe0-ce6da5483ec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f05d774edb4917bb4926dc03516d69', 'request_id': 'req-d3844511-a0c3-4214-afe0-ce6da5483ec5'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.850982 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d3844511-a0c3-4214-afe0-ce6da5483ec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f05d774edb4917bb4926dc03516d69', 'request_id': 'req-d3844511-a0c3-4214-afe0-ce6da5483ec5'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.963695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d3844511-a0c3-4214-afe0-ce6da5483ec5 admin admin] The token cache is being invalidate because project 56f05d774edb4917bb4926dc03516d69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:35.964004 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d3844511-a0c3-4214-afe0-ce6da5483ec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.964767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d3844511-a0c3-4214-afe0-ce6da5483ec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:35.966323 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 989/1993] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:35 2022] DELETE /identity/v3/projects/56f05d774edb4917bb4926dc03516d69 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:35.972614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f25d7-5909-4813-95ee-c59b4d63c114 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:35.973455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f25d7-5909-4813-95ee-c59b4d63c114 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:35.973780 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f25d7-5909-4813-95ee-c59b4d63c114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:35.995494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ad1f25d7-5909-4813-95ee-c59b4d63c114 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:35.995951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ad1f25d7-5909-4813-95ee-c59b4d63c114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:36.057506 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1005/1994] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:36.065418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700b6a-4c42-4f90-b142-7589fd578042 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.065688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700b6a-4c42-4f90-b142-7589fd578042 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.065988 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700b6a-4c42-4f90-b142-7589fd578042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.088559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-5c700b6a-4c42-4f90-b142-7589fd578042 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:36.089423 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-5c700b6a-4c42-4f90-b142-7589fd578042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:36.098412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 990/1995] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:36.102727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f25d7-5909-4813-95ee-c59b4d63c114 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:36.113896 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f25d7-5909-4813-95ee-c59b4d63c114 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:36.120119 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb3377-af38-4546-9d91-06fc19eb93c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:36.120778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb3377-af38-4546-9d91-06fc19eb93c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.121176 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb3377-af38-4546-9d91-06fc19eb93c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.121473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb3377-af38-4546-9d91-06fc19eb93c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.122264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb3377-af38-4546-9d91-06fc19eb93c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:49:36.122576 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb3377-af38-4546-9d91-06fc19eb93c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:36.124207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb3377-af38-4546-9d91-06fc19eb93c8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:36.137516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1006/1996] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:49:36.144613 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700b6a-4c42-4f90-b142-7589fd578042 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:36.153849 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700b6a-4c42-4f90-b142-7589fd578042 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:36.158423 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55296c-4db7-45d4-b0ff-1d809047dcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:36.159043 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55296c-4db7-45d4-b0ff-1d809047dcdc admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.159320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55296c-4db7-45d4-b0ff-1d809047dcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.159589 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55296c-4db7-45d4-b0ff-1d809047dcdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.160305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55296c-4db7-45d4-b0ff-1d809047dcdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:36.162291 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55296c-4db7-45d4-b0ff-1d809047dcdc admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:36.178144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb3377-af38-4546-9d91-06fc19eb93c8 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:36.180318 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e1bb3377-af38-4546-9d91-06fc19eb93c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:36.196155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb3377-af38-4546-9d91-06fc19eb93c8 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:36.206974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5789c-3584-454b-8eed-485532716fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:36.206974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5789c-3584-454b-8eed-485532716fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.206974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5789c-3584-454b-8eed-485532716fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.206974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5789c-3584-454b-8eed-485532716fb8 admin admin] PATH_INFO: `/v3/projects/d23928ad23834d199ddf10e3e445771f` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.207329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5789c-3584-454b-8eed-485532716fb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d23928ad23834d199ddf10e3e445771f)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:36.207329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5789c-3584-454b-8eed-485532716fb8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:36.265882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 991/1997] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:36 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:36.269733 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55296c-4db7-45d4-b0ff-1d809047dcdc admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:36.331628 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55296c-4db7-45d4-b0ff-1d809047dcdc admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:36.342614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-57c5789c-3584-454b-8eed-485532716fb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd23928ad23834d199ddf10e3e445771f', 'request_id': 'req-57c5789c-3584-454b-8eed-485532716fb8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:36.349887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-57c5789c-3584-454b-8eed-485532716fb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd23928ad23834d199ddf10e3e445771f', 'request_id': 'req-57c5789c-3584-454b-8eed-485532716fb8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:36.356321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-57c5789c-3584-454b-8eed-485532716fb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd23928ad23834d199ddf10e3e445771f', 'request_id': 'req-57c5789c-3584-454b-8eed-485532716fb8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:36.388340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ff79a-1d2b-4be1-9944-f6fa0b50d63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:36.388976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ff79a-1d2b-4be1-9944-f6fa0b50d63a admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.389260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ff79a-1d2b-4be1-9944-f6fa0b50d63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.389533 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ff79a-1d2b-4be1-9944-f6fa0b50d63a admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.390149 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ff79a-1d2b-4be1-9944-f6fa0b50d63a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:36.392043 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ff79a-1d2b-4be1-9944-f6fa0b50d63a admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:36.441250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-57c5789c-3584-454b-8eed-485532716fb8 admin admin] The token cache is being invalidate because project d23928ad23834d199ddf10e3e445771f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:36.441792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-57c5789c-3584-454b-8eed-485532716fb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:36.442481 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-57c5789c-3584-454b-8eed-485532716fb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:36.444121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1007/1998] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:36 2022] DELETE /identity/v3/projects/d23928ad23834d199ddf10e3e445771f => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:36.449183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547d90c-3e10-446e-8d36-d6222ec58a0d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.449473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547d90c-3e10-446e-8d36-d6222ec58a0d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.449750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547d90c-3e10-446e-8d36-d6222ec58a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.468725 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0547d90c-3e10-446e-8d36-d6222ec58a0d None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:36.471073 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:36.471840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0547d90c-3e10-446e-8d36-d6222ec58a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:36.480248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1008/1999] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:36.487833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728a950-328a-4517-898f-21ac1d71b2b3 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.488120 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728a950-328a-4517-898f-21ac1d71b2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.488395 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728a950-328a-4517-898f-21ac1d71b2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.507451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9728a950-328a-4517-898f-21ac1d71b2b3 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:36.508642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9728a950-328a-4517-898f-21ac1d71b2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:36.518937 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 992/2000] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:36 2022] POST /identity/v3/users => generated 292 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:36.519801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1009/2001] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:36.520096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ff79a-1d2b-4be1-9944-f6fa0b50d63a admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:36.520760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728a950-328a-4517-898f-21ac1d71b2b3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:36.521954 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-044ff79a-1d2b-4be1-9944-f6fa0b50d63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:36.530857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728a950-328a-4517-898f-21ac1d71b2b3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:36.535375 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ff79a-1d2b-4be1-9944-f6fa0b50d63a admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:36.535691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76beb300-0c2a-4799-84e7-b24990cc8522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:36.536310 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beb300-0c2a-4799-84e7-b24990cc8522 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.536627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beb300-0c2a-4799-84e7-b24990cc8522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.536894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beb300-0c2a-4799-84e7-b24990cc8522 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.537558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76beb300-0c2a-4799-84e7-b24990cc8522 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:49:36.537865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76beb300-0c2a-4799-84e7-b24990cc8522 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:36.539459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76beb300-0c2a-4799-84e7-b24990cc8522 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:36.540740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749ca4b-243e-43bd-8a0d-acc6c56949c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:36.542287 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749ca4b-243e-43bd-8a0d-acc6c56949c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.544172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749ca4b-243e-43bd-8a0d-acc6c56949c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.544172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749ca4b-243e-43bd-8a0d-acc6c56949c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.544172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749ca4b-243e-43bd-8a0d-acc6c56949c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:36.545626 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749ca4b-243e-43bd-8a0d-acc6c56949c2 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:36.558925 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1010/2002] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:49:36.559158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 993/2003] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:36.559594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76beb300-0c2a-4799-84e7-b24990cc8522 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:36.560150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749ca4b-243e-43bd-8a0d-acc6c56949c2 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:36.570456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76beb300-0c2a-4799-84e7-b24990cc8522 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:36.574228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749ca4b-243e-43bd-8a0d-acc6c56949c2 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:36.575444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286898e5-afa9-4e7a-9b49-ce27dc25b533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:36.576052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286898e5-afa9-4e7a-9b49-ce27dc25b533 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.576335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286898e5-afa9-4e7a-9b49-ce27dc25b533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.576753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286898e5-afa9-4e7a-9b49-ce27dc25b533 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.577359 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286898e5-afa9-4e7a-9b49-ce27dc25b533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:36.579151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90045f-0982-4f5a-a489-fdd075986778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:36.579272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286898e5-afa9-4e7a-9b49-ce27dc25b533 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:36.579794 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90045f-0982-4f5a-a489-fdd075986778 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.580080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90045f-0982-4f5a-a489-fdd075986778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.580359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90045f-0982-4f5a-a489-fdd075986778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.581043 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90045f-0982-4f5a-a489-fdd075986778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:36.582633 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90045f-0982-4f5a-a489-fdd075986778 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:36.608630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 994/2004] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:36.612480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90045f-0982-4f5a-a489-fdd075986778 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:36.622544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90045f-0982-4f5a-a489-fdd075986778 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:36.627297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb376e-5ec6-4ce4-9e83-d1168eba58fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:36.627921 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb376e-5ec6-4ce4-9e83-d1168eba58fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.628199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb376e-5ec6-4ce4-9e83-d1168eba58fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.628506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb376e-5ec6-4ce4-9e83-d1168eba58fb admin admin] PATH_INFO: `/v3/projects/f3356034de4b4c4db10ecafff6d61a63/users/301c164d761249b2adcae20251579fe3/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.644607 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb376e-5ec6-4ce4-9e83-d1168eba58fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3356034de4b4c4db10ecafff6d61a63, user_id=301c164d761249b2adcae20251579fe3, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:36.647551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb376e-5ec6-4ce4-9e83-d1168eba58fb admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:36.651788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1011/2005] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:36 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:36.656009 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286898e5-afa9-4e7a-9b49-ce27dc25b533 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:36.715387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286898e5-afa9-4e7a-9b49-ce27dc25b533 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:36.720071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e8199-fd80-4310-9db9-64263159756a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:36.720726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e8199-fd80-4310-9db9-64263159756a admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.721011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e8199-fd80-4310-9db9-64263159756a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.721290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e8199-fd80-4310-9db9-64263159756a admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.721921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e8199-fd80-4310-9db9-64263159756a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:36.723879 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e8199-fd80-4310-9db9-64263159756a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:36.727274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 995/2006] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:36 2022] PUT /identity/v3/projects/f3356034de4b4c4db10ecafff6d61a63/users/301c164d761249b2adcae20251579fe3/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:36.738967 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded90af-cf2f-47db-a7e8-dc6007fafb26 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.739272 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded90af-cf2f-47db-a7e8-dc6007fafb26 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.739549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded90af-cf2f-47db-a7e8-dc6007fafb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.769730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-0ded90af-cf2f-47db-a7e8-dc6007fafb26 None None] MFA Rules not processed for user `301c164d761249b2adcae20251579fe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:36.771074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-0ded90af-cf2f-47db-a7e8-dc6007fafb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:36.808685 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1012/2007] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:36 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:36.812826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e8199-fd80-4310-9db9-64263159756a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:36.827965 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e8199-fd80-4310-9db9-64263159756a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:36.847629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 996/2008] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:36.860561 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded90af-cf2f-47db-a7e8-dc6007fafb26 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:36.863041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:36.864983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-0ded90af-cf2f-47db-a7e8-dc6007fafb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:36.893600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2cdc9-3a5d-466e-8255-3cffacabab50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:36.894229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2cdc9-3a5d-466e-8255-3cffacabab50 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.894546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2cdc9-3a5d-466e-8255-3cffacabab50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.894832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2cdc9-3a5d-466e-8255-3cffacabab50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.895518 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2cdc9-3a5d-466e-8255-3cffacabab50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:36.897248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2cdc9-3a5d-466e-8255-3cffacabab50 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:36.911031 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1013/2009] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:49:36.912068 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2cdc9-3a5d-466e-8255-3cffacabab50 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:36.923829 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2cdc9-3a5d-466e-8255-3cffacabab50 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:36.928704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c56102-514f-42e9-93ef-19b589c43bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:36.929310 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c56102-514f-42e9-93ef-19b589c43bed admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.929594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c56102-514f-42e9-93ef-19b589c43bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.929866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c56102-514f-42e9-93ef-19b589c43bed admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.930519 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c56102-514f-42e9-93ef-19b589c43bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:36.931398 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded90af-cf2f-47db-a7e8-dc6007fafb26 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:36.932334 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c56102-514f-42e9-93ef-19b589c43bed admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:36.935939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc7a24-78b4-4560-8b49-9b4f135478a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:36.936705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc7a24-78b4-4560-8b49-9b4f135478a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.937090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc7a24-78b4-4560-8b49-9b4f135478a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.937390 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc7a24-78b4-4560-8b49-9b4f135478a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.948572 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1014/2010] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:36.949589 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc7a24-78b4-4560-8b49-9b4f135478a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:36.949949 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c56102-514f-42e9-93ef-19b589c43bed admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:36.951062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc7a24-78b4-4560-8b49-9b4f135478a4 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:36.959713 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c56102-514f-42e9-93ef-19b589c43bed admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:36.960712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 997/2011] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:49:36.965305 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19cdb5-ab6f-41a7-88d8-808608f4b542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:36.965990 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19cdb5-ab6f-41a7-88d8-808608f4b542 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:36.966222 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19cdb5-ab6f-41a7-88d8-808608f4b542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:36.966493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19cdb5-ab6f-41a7-88d8-808608f4b542 admin admin] PATH_INFO: `/v3/projects/24aba6abafc544bf9dcb22c120105de3/users/fac76c44356045f6a963394c7720ed8f/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:36.981313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19cdb5-ab6f-41a7-88d8-808608f4b542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24aba6abafc544bf9dcb22c120105de3, user_id=fac76c44356045f6a963394c7720ed8f, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:36.987304 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19cdb5-ab6f-41a7-88d8-808608f4b542 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:37.065350 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1015/2012] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:36 2022] PUT /identity/v3/projects/24aba6abafc544bf9dcb22c120105de3/users/fac76c44356045f6a963394c7720ed8f/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:37.077306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593edf7-4155-4bcc-809f-24cb52cfe30d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:37.077593 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593edf7-4155-4bcc-809f-24cb52cfe30d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:37.077872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593edf7-4155-4bcc-809f-24cb52cfe30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:37.105682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-1593edf7-4155-4bcc-809f-24cb52cfe30d None None] MFA Rules not processed for user `fac76c44356045f6a963394c7720ed8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:37.106912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1593edf7-4155-4bcc-809f-24cb52cfe30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:37.181682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 998/2013] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:37.187857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19cdb5-ab6f-41a7-88d8-808608f4b542 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:37.256213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19cdb5-ab6f-41a7-88d8-808608f4b542 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:37.266882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacb968-2d36-4797-b3cb-c3023a1863a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:37.267766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacb968-2d36-4797-b3cb-c3023a1863a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:37.268977 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacb968-2d36-4797-b3cb-c3023a1863a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:37.269367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacb968-2d36-4797-b3cb-c3023a1863a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:37.271570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacb968-2d36-4797-b3cb-c3023a1863a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:37.273637 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacb968-2d36-4797-b3cb-c3023a1863a0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:37.371228 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1016/2014] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:37 2022] POST /identity/v3/projects => generated 387 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:37.375407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593edf7-4155-4bcc-809f-24cb52cfe30d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:37.403187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593edf7-4155-4bcc-809f-24cb52cfe30d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:37.447445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:37.460208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2484dc1-6ddc-4638-ae0e-c92c367a6397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:37.460848 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2484dc1-6ddc-4638-ae0e-c92c367a6397 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:37.461132 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2484dc1-6ddc-4638-ae0e-c92c367a6397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:37.461408 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2484dc1-6ddc-4638-ae0e-c92c367a6397 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:37.462041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2484dc1-6ddc-4638-ae0e-c92c367a6397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:37.464062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2484dc1-6ddc-4638-ae0e-c92c367a6397 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:37.588881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 999/2015] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:37 2022] POST /identity/v3/users => generated 300 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:37.590046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacb968-2d36-4797-b3cb-c3023a1863a0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:37.600737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacb968-2d36-4797-b3cb-c3023a1863a0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:37.606031 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef180a02-72c8-4f0d-ae57-a2005686133f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:37.606635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef180a02-72c8-4f0d-ae57-a2005686133f admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:37.606916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef180a02-72c8-4f0d-ae57-a2005686133f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:37.607189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef180a02-72c8-4f0d-ae57-a2005686133f admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:37.609269 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef180a02-72c8-4f0d-ae57-a2005686133f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:37.609583 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef180a02-72c8-4f0d-ae57-a2005686133f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:37.626751 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1017/2016] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:37.629686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2484dc1-6ddc-4638-ae0e-c92c367a6397 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:37.640988 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2484dc1-6ddc-4638-ae0e-c92c367a6397 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:37.646361 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9ae19-2533-4f4a-8a0f-1721a4ed7d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:37.648706 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9ae19-2533-4f4a-8a0f-1721a4ed7d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:37.648706 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9ae19-2533-4f4a-8a0f-1721a4ed7d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:37.648706 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9ae19-2533-4f4a-8a0f-1721a4ed7d05 admin admin] PATH_INFO: `/v3/projects/2d895d0d3747434389c638f7a6299c3d/users/bc17d0e76de14d399912495ad6ffbcc6/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:37.672986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9ae19-2533-4f4a-8a0f-1721a4ed7d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d895d0d3747434389c638f7a6299c3d, user_id=bc17d0e76de14d399912495ad6ffbcc6, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:37.677634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9ae19-2533-4f4a-8a0f-1721a4ed7d05 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:37.752859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1000/2017] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:37 2022] PUT /identity/v3/projects/2d895d0d3747434389c638f7a6299c3d/users/bc17d0e76de14d399912495ad6ffbcc6/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:37.757021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef180a02-72c8-4f0d-ae57-a2005686133f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:37.767248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef180a02-72c8-4f0d-ae57-a2005686133f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:37.827760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf3b8c-ee80-4e09-8ed1-7cb38595d537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:37.828382 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf3b8c-ee80-4e09-8ed1-7cb38595d537 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:37.828697 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf3b8c-ee80-4e09-8ed1-7cb38595d537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:37.828969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf3b8c-ee80-4e09-8ed1-7cb38595d537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:37.829620 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf3b8c-ee80-4e09-8ed1-7cb38595d537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:37.831215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf3b8c-ee80-4e09-8ed1-7cb38595d537 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:37.841321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1018/2018] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:37.845621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9ae19-2533-4f4a-8a0f-1721a4ed7d05 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:37.852800 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:37.858398 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9ae19-2533-4f4a-8a0f-1721a4ed7d05 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:37.863162 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c8854-fb24-4675-98cc-17dd5cd6e38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:37.863816 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c8854-fb24-4675-98cc-17dd5cd6e38e admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:37.864100 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c8854-fb24-4675-98cc-17dd5cd6e38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:37.864377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c8854-fb24-4675-98cc-17dd5cd6e38e admin admin] PATH_INFO: `/v3/projects/2d895d0d3747434389c638f7a6299c3d/users/bc17d0e76de14d399912495ad6ffbcc6/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:37.869291 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c8854-fb24-4675-98cc-17dd5cd6e38e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d895d0d3747434389c638f7a6299c3d, user_id=bc17d0e76de14d399912495ad6ffbcc6, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:37.870001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c8854-fb24-4675-98cc-17dd5cd6e38e admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:37.880598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1001/2019] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:37 2022] PUT /identity/v3/projects/2d895d0d3747434389c638f7a6299c3d/users/bc17d0e76de14d399912495ad6ffbcc6/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:37.890906 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47179f18-8718-422a-8573-2f89e93e6c39 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:37.891240 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47179f18-8718-422a-8573-2f89e93e6c39 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:37.891540 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47179f18-8718-422a-8573-2f89e93e6c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:37.919736 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-47179f18-8718-422a-8573-2f89e93e6c39 None None] MFA Rules not processed for user `bc17d0e76de14d399912495ad6ffbcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:37.921151 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-47179f18-8718-422a-8573-2f89e93e6c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:37.994129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1019/2020] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:38.006225 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c8854-fb24-4675-98cc-17dd5cd6e38e admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:38.007922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-042c8854-fb24-4675-98cc-17dd5cd6e38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:38.027976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:38.057732 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:38.070361 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c8854-fb24-4675-98cc-17dd5cd6e38e admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:38.074858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc47f20-2e56-4bdc-a717-5193033f50d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:38.075472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc47f20-2e56-4bdc-a717-5193033f50d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:38.075757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc47f20-2e56-4bdc-a717-5193033f50d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:38.076032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc47f20-2e56-4bdc-a717-5193033f50d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:38.085677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc47f20-2e56-4bdc-a717-5193033f50d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:38.087283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc47f20-2e56-4bdc-a717-5193033f50d1 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:38.097480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1002/2021] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:49:38.161624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47179f18-8718-422a-8573-2f89e93e6c39 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:38.223124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47179f18-8718-422a-8573-2f89e93e6c39 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:38.228343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8814b0-c8b5-41c2-8726-809677360819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:38.229579 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8814b0-c8b5-41c2-8726-809677360819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:38.229858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8814b0-c8b5-41c2-8726-809677360819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:38.230130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8814b0-c8b5-41c2-8726-809677360819 admin admin] PATH_INFO: `/v3/users/fac76c44356045f6a963394c7720ed8f` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:38.231260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8814b0-c8b5-41c2-8726-809677360819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac76c44356045f6a963394c7720ed8f)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:38.233237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8814b0-c8b5-41c2-8726-809677360819 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:38.424952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5e8814b0-c8b5-41c2-8726-809677360819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac76c44356045f6a963394c7720ed8f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:38.425743 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5e8814b0-c8b5-41c2-8726-809677360819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac76c44356045f6a963394c7720ed8f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:38.426500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5e8814b0-c8b5-41c2-8726-809677360819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac76c44356045f6a963394c7720ed8f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:38.661865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5e8814b0-c8b5-41c2-8726-809677360819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac76c44356045f6a963394c7720ed8f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:38.668691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5e8814b0-c8b5-41c2-8726-809677360819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac76c44356045f6a963394c7720ed8f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:38.688117 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1020/2022] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:38 2022] DELETE /identity/v3/users/fac76c44356045f6a963394c7720ed8f => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:38.692437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7ce31-42f0-4c50-915b-3dbeedf399df None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:38.692938 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7ce31-42f0-4c50-915b-3dbeedf399df None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:38.693074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7ce31-42f0-4c50-915b-3dbeedf399df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:38.714840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-ecf7ce31-42f0-4c50-915b-3dbeedf399df None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:38.716070 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ecf7ce31-42f0-4c50-915b-3dbeedf399df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:38.778344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1003/2023] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:38.792302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8814b0-c8b5-41c2-8726-809677360819 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:38.793868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5e8814b0-c8b5-41c2-8726-809677360819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:38.817668 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:38.855483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8814b0-c8b5-41c2-8726-809677360819 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:38.859029 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5903c38-9afb-46c4-9db2-8364fe4b7b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:38.859631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5903c38-9afb-46c4-9db2-8364fe4b7b87 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:38.859908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5903c38-9afb-46c4-9db2-8364fe4b7b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:38.860180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5903c38-9afb-46c4-9db2-8364fe4b7b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:38.869262 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5903c38-9afb-46c4-9db2-8364fe4b7b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:38.871205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5903c38-9afb-46c4-9db2-8364fe4b7b87 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:38.880669 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1021/2024] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:39.283463 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7ce31-42f0-4c50-915b-3dbeedf399df None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:39.291404 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ecf7ce31-42f0-4c50-915b-3dbeedf399df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:39.298317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7ce31-42f0-4c50-915b-3dbeedf399df None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:39.304189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec65d6-3283-4f14-906d-84fb67d9d1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:39.304879 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec65d6-3283-4f14-906d-84fb67d9d1df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:39.305159 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec65d6-3283-4f14-906d-84fb67d9d1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:39.305435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec65d6-3283-4f14-906d-84fb67d9d1df admin admin] PATH_INFO: `/v3/projects/24aba6abafc544bf9dcb22c120105de3` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:39.313873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec65d6-3283-4f14-906d-84fb67d9d1df admin admin] RBAC: Authorizing `identity:delete_project(project_id=24aba6abafc544bf9dcb22c120105de3)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:39.315863 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec65d6-3283-4f14-906d-84fb67d9d1df admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:39.438624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2dec65d6-3283-4f14-906d-84fb67d9d1df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24aba6abafc544bf9dcb22c120105de3', 'request_id': 'req-2dec65d6-3283-4f14-906d-84fb67d9d1df'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:39.444905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2dec65d6-3283-4f14-906d-84fb67d9d1df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24aba6abafc544bf9dcb22c120105de3', 'request_id': 'req-2dec65d6-3283-4f14-906d-84fb67d9d1df'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:39.450536 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2dec65d6-3283-4f14-906d-84fb67d9d1df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24aba6abafc544bf9dcb22c120105de3', 'request_id': 'req-2dec65d6-3283-4f14-906d-84fb67d9d1df'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:39.541243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2dec65d6-3283-4f14-906d-84fb67d9d1df admin admin] The token cache is being invalidate because project 24aba6abafc544bf9dcb22c120105de3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:39.543556 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2dec65d6-3283-4f14-906d-84fb67d9d1df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:39.543556 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2dec65d6-3283-4f14-906d-84fb67d9d1df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:39.546441 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1004/2025] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:39 2022] DELETE /identity/v3/projects/24aba6abafc544bf9dcb22c120105de3 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:39.548862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5903c38-9afb-46c4-9db2-8364fe4b7b87 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:39.560837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5903c38-9afb-46c4-9db2-8364fe4b7b87 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:39.619088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81021dd8-921a-4fea-957b-26bb3b60bb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:39.619751 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81021dd8-921a-4fea-957b-26bb3b60bb1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:39.620033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81021dd8-921a-4fea-957b-26bb3b60bb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:39.620308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81021dd8-921a-4fea-957b-26bb3b60bb1a admin admin] PATH_INFO: `/v3/users/bc17d0e76de14d399912495ad6ffbcc6` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:39.621571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81021dd8-921a-4fea-957b-26bb3b60bb1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc17d0e76de14d399912495ad6ffbcc6)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:39.623625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81021dd8-921a-4fea-957b-26bb3b60bb1a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:39.808989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-81021dd8-921a-4fea-957b-26bb3b60bb1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc17d0e76de14d399912495ad6ffbcc6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:39.810063 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-81021dd8-921a-4fea-957b-26bb3b60bb1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc17d0e76de14d399912495ad6ffbcc6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:39.813374 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-81021dd8-921a-4fea-957b-26bb3b60bb1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc17d0e76de14d399912495ad6ffbcc6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:39.907436 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-81021dd8-921a-4fea-957b-26bb3b60bb1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc17d0e76de14d399912495ad6ffbcc6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:39.915066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-81021dd8-921a-4fea-957b-26bb3b60bb1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc17d0e76de14d399912495ad6ffbcc6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:39.935112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1022/2026] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:39 2022] DELETE /identity/v3/users/bc17d0e76de14d399912495ad6ffbcc6 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:40.446158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec65d6-3283-4f14-906d-84fb67d9d1df admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:40.448804 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2dec65d6-3283-4f14-906d-84fb67d9d1df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:40.462740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec65d6-3283-4f14-906d-84fb67d9d1df admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:40.523680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7ab61-5702-4bfb-93bf-6c20ce02c91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:40.524307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7ab61-5702-4bfb-93bf-6c20ce02c91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:40.524869 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7ab61-5702-4bfb-93bf-6c20ce02c91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:40.524869 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7ab61-5702-4bfb-93bf-6c20ce02c91e admin admin] PATH_INFO: `/v3/projects/2d895d0d3747434389c638f7a6299c3d` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:40.525996 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7ab61-5702-4bfb-93bf-6c20ce02c91e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d895d0d3747434389c638f7a6299c3d)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:40.527894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7ab61-5702-4bfb-93bf-6c20ce02c91e admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:40.534023 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:40.644465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c6f7ab61-5702-4bfb-93bf-6c20ce02c91e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d895d0d3747434389c638f7a6299c3d', 'request_id': 'req-c6f7ab61-5702-4bfb-93bf-6c20ce02c91e'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:40.650618 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c6f7ab61-5702-4bfb-93bf-6c20ce02c91e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d895d0d3747434389c638f7a6299c3d', 'request_id': 'req-c6f7ab61-5702-4bfb-93bf-6c20ce02c91e'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:40.655996 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c6f7ab61-5702-4bfb-93bf-6c20ce02c91e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d895d0d3747434389c638f7a6299c3d', 'request_id': 'req-c6f7ab61-5702-4bfb-93bf-6c20ce02c91e'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:40.721041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c6f7ab61-5702-4bfb-93bf-6c20ce02c91e admin admin] The token cache is being invalidate because project 2d895d0d3747434389c638f7a6299c3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:40.721325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c6f7ab61-5702-4bfb-93bf-6c20ce02c91e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:40.721967 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c6f7ab61-5702-4bfb-93bf-6c20ce02c91e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:40.723501 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1005/2027] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:40 2022] DELETE /identity/v3/projects/2d895d0d3747434389c638f7a6299c3d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:40.728448 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff90d6e-5394-467c-a77a-27955d87b2c8 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:40.729424 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff90d6e-5394-467c-a77a-27955d87b2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:40.729839 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff90d6e-5394-467c-a77a-27955d87b2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:40.753555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fff90d6e-5394-467c-a77a-27955d87b2c8 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:40.754847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fff90d6e-5394-467c-a77a-27955d87b2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:40.787446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:40.818278 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1023/2028] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:40.826051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90520d7-f6d5-4462-8dc6-e71a24f8b8ac None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:40.826335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90520d7-f6d5-4462-8dc6-e71a24f8b8ac None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:40.826605 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90520d7-f6d5-4462-8dc6-e71a24f8b8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:40.847532 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-a90520d7-f6d5-4462-8dc6-e71a24f8b8ac None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:40.849718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a90520d7-f6d5-4462-8dc6-e71a24f8b8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:40.859487 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1006/2029] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:40.862626 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff90d6e-5394-467c-a77a-27955d87b2c8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:40.873077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff90d6e-5394-467c-a77a-27955d87b2c8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:40.877682 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e28cf0-b6e9-44ba-942e-6afaf8a51923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:40.883186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e28cf0-b6e9-44ba-942e-6afaf8a51923 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:40.883186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e28cf0-b6e9-44ba-942e-6afaf8a51923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:40.883186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e28cf0-b6e9-44ba-942e-6afaf8a51923 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:40.883186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e28cf0-b6e9-44ba-942e-6afaf8a51923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:49:40.883186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e28cf0-b6e9-44ba-942e-6afaf8a51923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:40.883186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e28cf0-b6e9-44ba-942e-6afaf8a51923 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:40.892623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1024/2030] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:49:40.897447 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90520d7-f6d5-4462-8dc6-e71a24f8b8ac None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:40.909638 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90520d7-f6d5-4462-8dc6-e71a24f8b8ac None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:40.915221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87228332-ae5c-42df-a27c-fa9bb9c2b827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:40.915918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87228332-ae5c-42df-a27c-fa9bb9c2b827 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:40.916837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87228332-ae5c-42df-a27c-fa9bb9c2b827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:40.918748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87228332-ae5c-42df-a27c-fa9bb9c2b827 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:40.918748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87228332-ae5c-42df-a27c-fa9bb9c2b827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:40.920370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87228332-ae5c-42df-a27c-fa9bb9c2b827 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:41.037051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1007/2031] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:40 2022] POST /identity/v3/projects => generated 391 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:41.037307 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e28cf0-b6e9-44ba-942e-6afaf8a51923 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:41.048339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e28cf0-b6e9-44ba-942e-6afaf8a51923 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:41.115432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fef6cd-5b14-4d94-b89b-d71098b83028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:41.116316 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fef6cd-5b14-4d94-b89b-d71098b83028 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:41.117241 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fef6cd-5b14-4d94-b89b-d71098b83028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:41.117978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fef6cd-5b14-4d94-b89b-d71098b83028 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:41.119527 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fef6cd-5b14-4d94-b89b-d71098b83028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:41.122712 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fef6cd-5b14-4d94-b89b-d71098b83028 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:41.200543 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87228332-ae5c-42df-a27c-fa9bb9c2b827 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:41.202274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-87228332-ae5c-42df-a27c-fa9bb9c2b827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:41.207624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1025/2032] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:41 2022] POST /identity/v3/users => generated 296 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:41.215447 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87228332-ae5c-42df-a27c-fa9bb9c2b827 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:41.216152 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fef6cd-5b14-4d94-b89b-d71098b83028 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:41.220229 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bba134-bc26-41ea-bab9-a81e7801b119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:41.220985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bba134-bc26-41ea-bab9-a81e7801b119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:41.221269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bba134-bc26-41ea-bab9-a81e7801b119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:41.221542 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bba134-bc26-41ea-bab9-a81e7801b119 admin admin] PATH_INFO: `/v3/users/301c164d761249b2adcae20251579fe3` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:41.222679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bba134-bc26-41ea-bab9-a81e7801b119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=301c164d761249b2adcae20251579fe3)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:41.224641 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bba134-bc26-41ea-bab9-a81e7801b119 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:41.226098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fef6cd-5b14-4d94-b89b-d71098b83028 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:41.231594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85d912-0701-4729-aa1b-312ca3c3374d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:41.232192 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85d912-0701-4729-aa1b-312ca3c3374d admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:41.232675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85d912-0701-4729-aa1b-312ca3c3374d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:41.233206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85d912-0701-4729-aa1b-312ca3c3374d admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:41.234246 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85d912-0701-4729-aa1b-312ca3c3374d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:41.236477 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85d912-0701-4729-aa1b-312ca3c3374d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:41.249214 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1026/2033] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:49:41.253275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85d912-0701-4729-aa1b-312ca3c3374d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:41.263446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85d912-0701-4729-aa1b-312ca3c3374d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:41.268182 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fa031-d375-43b0-8cbb-022c35f16b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:41.268796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fa031-d375-43b0-8cbb-022c35f16b71 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:41.269078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fa031-d375-43b0-8cbb-022c35f16b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:41.269354 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fa031-d375-43b0-8cbb-022c35f16b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:41.269988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fa031-d375-43b0-8cbb-022c35f16b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:41.271571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fa031-d375-43b0-8cbb-022c35f16b71 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:41.281272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1027/2034] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:41.285201 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fa031-d375-43b0-8cbb-022c35f16b71 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:41.294894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fa031-d375-43b0-8cbb-022c35f16b71 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:41.299568 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7d025-3586-4bbd-8236-b179d3e10b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:41.300192 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7d025-3586-4bbd-8236-b179d3e10b5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:41.300508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7d025-3586-4bbd-8236-b179d3e10b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:41.300792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7d025-3586-4bbd-8236-b179d3e10b5f admin admin] PATH_INFO: `/v3/projects/273cfc672d354fdaa8e6c7d697e0fc95/users/8ebfd7c626ef40109f4aa79a342f800e/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:41.317586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7d025-3586-4bbd-8236-b179d3e10b5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=273cfc672d354fdaa8e6c7d697e0fc95, user_id=8ebfd7c626ef40109f4aa79a342f800e, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:41.320314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7d025-3586-4bbd-8236-b179d3e10b5f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:41.448914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1028/2035] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:41 2022] PUT /identity/v3/projects/273cfc672d354fdaa8e6c7d697e0fc95/users/8ebfd7c626ef40109f4aa79a342f800e/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:41.462349 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4cb78-ca71-4ae7-8665-ec65161449e0 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:41.462823 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4cb78-ca71-4ae7-8665-ec65161449e0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:41.462987 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4cb78-ca71-4ae7-8665-ec65161449e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:41.475301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a2bba134-bc26-41ea-bab9-a81e7801b119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '301c164d761249b2adcae20251579fe3'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:41.476524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a2bba134-bc26-41ea-bab9-a81e7801b119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '301c164d761249b2adcae20251579fe3'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:41.482628 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a2bba134-bc26-41ea-bab9-a81e7801b119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '301c164d761249b2adcae20251579fe3'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:41.487651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a2bba134-bc26-41ea-bab9-a81e7801b119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '301c164d761249b2adcae20251579fe3'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:41.496237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e6a4cb78-ca71-4ae7-8665-ec65161449e0 None None] MFA Rules not processed for user `8ebfd7c626ef40109f4aa79a342f800e`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:41.497494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e6a4cb78-ca71-4ae7-8665-ec65161449e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:41.558794 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a2bba134-bc26-41ea-bab9-a81e7801b119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '301c164d761249b2adcae20251579fe3'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:41.564232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:41.569388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1029/2036] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:41.573451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4cb78-ca71-4ae7-8665-ec65161449e0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:41.574821 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e6a4cb78-ca71-4ae7-8665-ec65161449e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:41.589338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1008/2037] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:41 2022] DELETE /identity/v3/users/301c164d761249b2adcae20251579fe3 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:41.593626 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d87805-fa4f-4189-9f8f-e1c3d3e8c998 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:41.594669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d87805-fa4f-4189-9f8f-e1c3d3e8c998 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:41.595066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d87805-fa4f-4189-9f8f-e1c3d3e8c998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:41.616734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-b9d87805-fa4f-4189-9f8f-e1c3d3e8c998 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:41.618183 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b9d87805-fa4f-4189-9f8f-e1c3d3e8c998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:41.650757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4cb78-ca71-4ae7-8665-ec65161449e0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:41.659899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:41.687700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1009/2038] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:41.698112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d87805-fa4f-4189-9f8f-e1c3d3e8c998 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:41.699577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b9d87805-fa4f-4189-9f8f-e1c3d3e8c998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:41.708715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b0af9-74ab-48ca-8a87-2743b058df01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:41.709560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b0af9-74ab-48ca-8a87-2743b058df01 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:41.709843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b0af9-74ab-48ca-8a87-2743b058df01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:41.710123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b0af9-74ab-48ca-8a87-2743b058df01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:41.710761 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b0af9-74ab-48ca-8a87-2743b058df01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:41.712704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b0af9-74ab-48ca-8a87-2743b058df01 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:41.758226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d87805-fa4f-4189-9f8f-e1c3d3e8c998 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:41.763328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e93ba-550d-482e-9ff9-a1c90298c801 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:41.763969 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e93ba-550d-482e-9ff9-a1c90298c801 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:41.764250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e93ba-550d-482e-9ff9-a1c90298c801 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:41.764563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e93ba-550d-482e-9ff9-a1c90298c801 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:41.774464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e93ba-550d-482e-9ff9-a1c90298c801 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:41.776519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e93ba-550d-482e-9ff9-a1c90298c801 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:41.786016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1010/2039] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:49:41.803363 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1030/2040] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:41 2022] POST /identity/v3/projects => generated 393 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:41.807562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e93ba-550d-482e-9ff9-a1c90298c801 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:41.817689 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e93ba-550d-482e-9ff9-a1c90298c801 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:41.881729 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e31d8-6046-4e5c-88c7-5110ee3c4fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:41.882554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e31d8-6046-4e5c-88c7-5110ee3c4fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:41.882932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e31d8-6046-4e5c-88c7-5110ee3c4fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:41.883269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e31d8-6046-4e5c-88c7-5110ee3c4fe5 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:41.885982 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e31d8-6046-4e5c-88c7-5110ee3c4fe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:41.886616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e31d8-6046-4e5c-88c7-5110ee3c4fe5 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:41.965918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1011/2041] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:41 2022] POST /identity/v3/users => generated 303 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:41.969255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b0af9-74ab-48ca-8a87-2743b058df01 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:41.980182 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b0af9-74ab-48ca-8a87-2743b058df01 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:41.984904 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e20c8-fb5e-44f7-8d33-f078dbd979d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:41.985514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e20c8-fb5e-44f7-8d33-f078dbd979d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:41.985849 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e20c8-fb5e-44f7-8d33-f078dbd979d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:41.986177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e20c8-fb5e-44f7-8d33-f078dbd979d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:41.986785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e20c8-fb5e-44f7-8d33-f078dbd979d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:41.988421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e20c8-fb5e-44f7-8d33-f078dbd979d4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:42.010786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1031/2042] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:42.017460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e31d8-6046-4e5c-88c7-5110ee3c4fe5 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:42.028471 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e31d8-6046-4e5c-88c7-5110ee3c4fe5 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:42.033649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a5b2c-39c1-4a27-90cb-4af2e2b7fa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:42.034425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a5b2c-39c1-4a27-90cb-4af2e2b7fa90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:42.034647 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a5b2c-39c1-4a27-90cb-4af2e2b7fa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:42.034923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a5b2c-39c1-4a27-90cb-4af2e2b7fa90 admin admin] PATH_INFO: `/v3/projects/6e7e5c34a5c94999a12fadeb80a72698/users/04d5d1975c874a63afa90162b7f7ffb6/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:42.050964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a5b2c-39c1-4a27-90cb-4af2e2b7fa90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e7e5c34a5c94999a12fadeb80a72698, user_id=04d5d1975c874a63afa90162b7f7ffb6, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:42.053977 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a5b2c-39c1-4a27-90cb-4af2e2b7fa90 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:42.122403 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1012/2043] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:42 2022] PUT /identity/v3/projects/6e7e5c34a5c94999a12fadeb80a72698/users/04d5d1975c874a63afa90162b7f7ffb6/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:42.126187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e20c8-fb5e-44f7-8d33-f078dbd979d4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:42.136642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e20c8-fb5e-44f7-8d33-f078dbd979d4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:42.208711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5932f-624d-493c-92c8-6ee4ef28f410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:42.209335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5932f-624d-493c-92c8-6ee4ef28f410 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:42.209622 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5932f-624d-493c-92c8-6ee4ef28f410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:42.209902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5932f-624d-493c-92c8-6ee4ef28f410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:42.212228 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5932f-624d-493c-92c8-6ee4ef28f410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:42.216204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5932f-624d-493c-92c8-6ee4ef28f410 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:42.219001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a5b2c-39c1-4a27-90cb-4af2e2b7fa90 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:42.220981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c35a5b2c-39c1-4a27-90cb-4af2e2b7fa90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:42.230507 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1032/2044] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:42.231336 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5932f-624d-493c-92c8-6ee4ef28f410 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:42.242768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5932f-624d-493c-92c8-6ee4ef28f410 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:42.245261 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a5b2c-39c1-4a27-90cb-4af2e2b7fa90 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:42.248340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c2105-8421-45e0-a302-550354a93d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:42.249047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c2105-8421-45e0-a302-550354a93d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:42.249472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c2105-8421-45e0-a302-550354a93d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:42.249756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c2105-8421-45e0-a302-550354a93d28 admin admin] PATH_INFO: `/v3/projects/6e7e5c34a5c94999a12fadeb80a72698/users/04d5d1975c874a63afa90162b7f7ffb6/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:42.252057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c2105-8421-45e0-a302-550354a93d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e7e5c34a5c94999a12fadeb80a72698, user_id=04d5d1975c874a63afa90162b7f7ffb6, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:42.252202 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5e6c3-a569-43e4-9a59-bf0400815f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:42.252682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5e6c3-a569-43e4-9a59-bf0400815f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:42.253029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5e6c3-a569-43e4-9a59-bf0400815f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:42.253999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5e6c3-a569-43e4-9a59-bf0400815f36 admin admin] PATH_INFO: `/v3/projects/f3356034de4b4c4db10ecafff6d61a63` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:42.254554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c2105-8421-45e0-a302-550354a93d28 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:42.254922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5e6c3-a569-43e4-9a59-bf0400815f36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3356034de4b4c4db10ecafff6d61a63)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:42.257757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5e6c3-a569-43e4-9a59-bf0400815f36 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:42.268775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1033/2045] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:42 2022] PUT /identity/v3/projects/6e7e5c34a5c94999a12fadeb80a72698/users/04d5d1975c874a63afa90162b7f7ffb6/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:42.279556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a5bce-9238-4af3-ae02-8536cd6822be None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:42.280107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a5bce-9238-4af3-ae02-8536cd6822be None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:42.280427 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a5bce-9238-4af3-ae02-8536cd6822be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:42.292449 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:42.313982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-d95a5bce-9238-4af3-ae02-8536cd6822be None None] MFA Rules not processed for user `04d5d1975c874a63afa90162b7f7ffb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:42.315327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d95a5bce-9238-4af3-ae02-8536cd6822be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:42.399320 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1034/2046] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:42 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:42.412277 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a5bce-9238-4af3-ae02-8536cd6822be None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:42.424796 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:42.425450 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d95a5bce-9238-4af3-ae02-8536cd6822be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:42.425742 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-67f5e6c3-a569-43e4-9a59-bf0400815f36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3356034de4b4c4db10ecafff6d61a63', 'request_id': 'req-67f5e6c3-a569-43e4-9a59-bf0400815f36'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:42.432775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-67f5e6c3-a569-43e4-9a59-bf0400815f36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3356034de4b4c4db10ecafff6d61a63', 'request_id': 'req-67f5e6c3-a569-43e4-9a59-bf0400815f36'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:42.439108 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-67f5e6c3-a569-43e4-9a59-bf0400815f36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3356034de4b4c4db10ecafff6d61a63', 'request_id': 'req-67f5e6c3-a569-43e4-9a59-bf0400815f36'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:42.489743 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a5bce-9238-4af3-ae02-8536cd6822be None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:42.512140 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-67f5e6c3-a569-43e4-9a59-bf0400815f36 admin admin] The token cache is being invalidate because project f3356034de4b4c4db10ecafff6d61a63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:42.512816 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-67f5e6c3-a569-43e4-9a59-bf0400815f36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:42.513130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-67f5e6c3-a569-43e4-9a59-bf0400815f36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:42.514696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1013/2047] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:42 2022] DELETE /identity/v3/projects/f3356034de4b4c4db10ecafff6d61a63 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:42.553515 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3dde29-be6f-4491-83d6-c9f1f27aaf6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:42.554158 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3dde29-be6f-4491-83d6-c9f1f27aaf6f service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:42.554440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3dde29-be6f-4491-83d6-c9f1f27aaf6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:42.554803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3dde29-be6f-4491-83d6-c9f1f27aaf6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:42.556804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-db3dde29-be6f-4491-83d6-c9f1f27aaf6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:42.644026 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:42.646787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3dde29-be6f-4491-83d6-c9f1f27aaf6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:42.649686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3dde29-be6f-4491-83d6-c9f1f27aaf6f service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:42.685135 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1035/2048] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:42 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:42.748284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5e6c3-a569-43e4-9a59-bf0400815f36 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:42.750699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-67f5e6c3-a569-43e4-9a59-bf0400815f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:42.766629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5e6c3-a569-43e4-9a59-bf0400815f36 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:42.770818 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3684094-ac73-45bc-900a-8a85284fbbef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:42.771504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3684094-ac73-45bc-900a-8a85284fbbef service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:42.771860 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3684094-ac73-45bc-900a-8a85284fbbef service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:42.772260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3684094-ac73-45bc-900a-8a85284fbbef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:42.775133 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d3684094-ac73-45bc-900a-8a85284fbbef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:42.855305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3684094-ac73-45bc-900a-8a85284fbbef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:42.856907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3684094-ac73-45bc-900a-8a85284fbbef service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:42.866983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1014/2049] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:49.033392 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3dde29-be6f-4491-83d6-c9f1f27aaf6f service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:49.035041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-db3dde29-be6f-4491-83d6-c9f1f27aaf6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:49.101790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3dde29-be6f-4491-83d6-c9f1f27aaf6f service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:49.106385 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95dd7a-7b1f-4f23-90b0-08d61d75e48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:49.107004 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95dd7a-7b1f-4f23-90b0-08d61d75e48a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:49.107287 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95dd7a-7b1f-4f23-90b0-08d61d75e48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:49.107563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95dd7a-7b1f-4f23-90b0-08d61d75e48a admin admin] PATH_INFO: `/v3/users/8ebfd7c626ef40109f4aa79a342f800e` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:49.108728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95dd7a-7b1f-4f23-90b0-08d61d75e48a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ebfd7c626ef40109f4aa79a342f800e)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:49.110662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95dd7a-7b1f-4f23-90b0-08d61d75e48a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:49.272037 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-dd95dd7a-7b1f-4f23-90b0-08d61d75e48a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ebfd7c626ef40109f4aa79a342f800e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:49.272891 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-dd95dd7a-7b1f-4f23-90b0-08d61d75e48a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ebfd7c626ef40109f4aa79a342f800e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:49.273650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-dd95dd7a-7b1f-4f23-90b0-08d61d75e48a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ebfd7c626ef40109f4aa79a342f800e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:49.352952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-dd95dd7a-7b1f-4f23-90b0-08d61d75e48a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ebfd7c626ef40109f4aa79a342f800e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:49.360714 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-dd95dd7a-7b1f-4f23-90b0-08d61d75e48a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ebfd7c626ef40109f4aa79a342f800e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:49.378870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1036/2050] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:49 2022] DELETE /identity/v3/users/8ebfd7c626ef40109f4aa79a342f800e => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:49.383560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28181814-c893-4a51-bfeb-9d2982b8502a None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:49.384047 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28181814-c893-4a51-bfeb-9d2982b8502a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:49.384142 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28181814-c893-4a51-bfeb-9d2982b8502a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:49.403973 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-28181814-c893-4a51-bfeb-9d2982b8502a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:49.405228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-28181814-c893-4a51-bfeb-9d2982b8502a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:49.477684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1015/2051] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:49.487626 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95dd7a-7b1f-4f23-90b0-08d61d75e48a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:49.489108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dd95dd7a-7b1f-4f23-90b0-08d61d75e48a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:49.553617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95dd7a-7b1f-4f23-90b0-08d61d75e48a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:49.558257 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a8f8c-37f0-4761-963a-6c01898b2fdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:49.558945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a8f8c-37f0-4761-963a-6c01898b2fdb service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:49.559380 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a8f8c-37f0-4761-963a-6c01898b2fdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:49.559698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a8f8c-37f0-4761-963a-6c01898b2fdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:49.569405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a8f8c-37f0-4761-963a-6c01898b2fdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:49.570840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a8f8c-37f0-4761-963a-6c01898b2fdb service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:49.580907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1037/2052] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:50.015610 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28181814-c893-4a51-bfeb-9d2982b8502a None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:50.017604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-28181814-c893-4a51-bfeb-9d2982b8502a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:50.046520 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28181814-c893-4a51-bfeb-9d2982b8502a None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:50.060076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778f5b9-c2c1-496c-b35b-972baa594edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:50.060792 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778f5b9-c2c1-496c-b35b-972baa594edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:50.061879 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778f5b9-c2c1-496c-b35b-972baa594edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:50.063759 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778f5b9-c2c1-496c-b35b-972baa594edc admin admin] PATH_INFO: `/v3/projects/273cfc672d354fdaa8e6c7d697e0fc95` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:50.063759 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778f5b9-c2c1-496c-b35b-972baa594edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=273cfc672d354fdaa8e6c7d697e0fc95)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:50.066373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778f5b9-c2c1-496c-b35b-972baa594edc admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:50.193457 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7778f5b9-c2c1-496c-b35b-972baa594edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '273cfc672d354fdaa8e6c7d697e0fc95', 'request_id': 'req-7778f5b9-c2c1-496c-b35b-972baa594edc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:50.201077 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7778f5b9-c2c1-496c-b35b-972baa594edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '273cfc672d354fdaa8e6c7d697e0fc95', 'request_id': 'req-7778f5b9-c2c1-496c-b35b-972baa594edc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:50.210603 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7778f5b9-c2c1-496c-b35b-972baa594edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '273cfc672d354fdaa8e6c7d697e0fc95', 'request_id': 'req-7778f5b9-c2c1-496c-b35b-972baa594edc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:50.213822 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:50.278361 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7778f5b9-c2c1-496c-b35b-972baa594edc admin admin] The token cache is being invalidate because project 273cfc672d354fdaa8e6c7d697e0fc95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:50.279092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7778f5b9-c2c1-496c-b35b-972baa594edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:50.280167 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7778f5b9-c2c1-496c-b35b-972baa594edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:50.282074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1016/2053] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:50 2022] DELETE /identity/v3/projects/273cfc672d354fdaa8e6c7d697e0fc95 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:50.286543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a8f8c-37f0-4761-963a-6c01898b2fdb service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:50.298322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a8f8c-37f0-4761-963a-6c01898b2fdb service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:50.317679 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:50.383441 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537b7e9-3c34-423e-8519-1620ce78d125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:50.384069 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537b7e9-3c34-423e-8519-1620ce78d125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:50.384289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537b7e9-3c34-423e-8519-1620ce78d125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:50.384613 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537b7e9-3c34-423e-8519-1620ce78d125 admin admin] PATH_INFO: `/v3/users/04d5d1975c874a63afa90162b7f7ffb6` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:50.387105 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537b7e9-3c34-423e-8519-1620ce78d125 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04d5d1975c874a63afa90162b7f7ffb6)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:50.389916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537b7e9-3c34-423e-8519-1620ce78d125 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:50.606957 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4537b7e9-3c34-423e-8519-1620ce78d125 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04d5d1975c874a63afa90162b7f7ffb6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:50.608329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4537b7e9-3c34-423e-8519-1620ce78d125 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04d5d1975c874a63afa90162b7f7ffb6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:50.608986 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4537b7e9-3c34-423e-8519-1620ce78d125 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04d5d1975c874a63afa90162b7f7ffb6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:50.687302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4537b7e9-3c34-423e-8519-1620ce78d125 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04d5d1975c874a63afa90162b7f7ffb6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:50.694875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4537b7e9-3c34-423e-8519-1620ce78d125 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04d5d1975c874a63afa90162b7f7ffb6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:50.714405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1038/2054] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:50 2022] DELETE /identity/v3/users/04d5d1975c874a63afa90162b7f7ffb6 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:51.400327 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778f5b9-c2c1-496c-b35b-972baa594edc admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:51.402733 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7778f5b9-c2c1-496c-b35b-972baa594edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:51.447732 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778f5b9-c2c1-496c-b35b-972baa594edc admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:51.525045 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef387ea-772c-4a6c-bb7e-7141501f78c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:51.525684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef387ea-772c-4a6c-bb7e-7141501f78c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:51.530918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef387ea-772c-4a6c-bb7e-7141501f78c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:51.530918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef387ea-772c-4a6c-bb7e-7141501f78c0 admin admin] PATH_INFO: `/v3/projects/6e7e5c34a5c94999a12fadeb80a72698` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:51.530918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef387ea-772c-4a6c-bb7e-7141501f78c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e7e5c34a5c94999a12fadeb80a72698)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:51.530918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef387ea-772c-4a6c-bb7e-7141501f78c0 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:51.682479 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-bef387ea-772c-4a6c-bb7e-7141501f78c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e7e5c34a5c94999a12fadeb80a72698', 'request_id': 'req-bef387ea-772c-4a6c-bb7e-7141501f78c0'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:51.689192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-bef387ea-772c-4a6c-bb7e-7141501f78c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e7e5c34a5c94999a12fadeb80a72698', 'request_id': 'req-bef387ea-772c-4a6c-bb7e-7141501f78c0'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:51.695114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-bef387ea-772c-4a6c-bb7e-7141501f78c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e7e5c34a5c94999a12fadeb80a72698', 'request_id': 'req-bef387ea-772c-4a6c-bb7e-7141501f78c0'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:51.783047 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-bef387ea-772c-4a6c-bb7e-7141501f78c0 admin admin] The token cache is being invalidate because project 6e7e5c34a5c94999a12fadeb80a72698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:51.783047 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-bef387ea-772c-4a6c-bb7e-7141501f78c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:51.783844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-bef387ea-772c-4a6c-bb7e-7141501f78c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:51.785905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1017/2055] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:51 2022] DELETE /identity/v3/projects/6e7e5c34a5c94999a12fadeb80a72698 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:51.790598 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6854196-1141-4518-bc05-b09c52aea1fe None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:51.791130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6854196-1141-4518-bc05-b09c52aea1fe None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:51.791130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6854196-1141-4518-bc05-b09c52aea1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:51.811147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e6854196-1141-4518-bc05-b09c52aea1fe None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:51.818532 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e6854196-1141-4518-bc05-b09c52aea1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:51.884933 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1039/2056] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:51.894468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686ccc6-6c83-44ca-b72a-d8ca4dec2dad None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:51.894837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686ccc6-6c83-44ca-b72a-d8ca4dec2dad None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:51.895177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686ccc6-6c83-44ca-b72a-d8ca4dec2dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:51.915191 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-7686ccc6-6c83-44ca-b72a-d8ca4dec2dad None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:51.916543 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7686ccc6-6c83-44ca-b72a-d8ca4dec2dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:51.925070 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1018/2057] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:51.929440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6854196-1141-4518-bc05-b09c52aea1fe None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:51.939308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6854196-1141-4518-bc05-b09c52aea1fe None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:51.943965 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1d5e2-8e76-4637-91bd-34060947f0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:51.944586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1d5e2-8e76-4637-91bd-34060947f0cd admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:51.944864 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1d5e2-8e76-4637-91bd-34060947f0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:51.945135 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1d5e2-8e76-4637-91bd-34060947f0cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:51.945781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1d5e2-8e76-4637-91bd-34060947f0cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:49:51.946107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1d5e2-8e76-4637-91bd-34060947f0cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:51.948284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1d5e2-8e76-4637-91bd-34060947f0cd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:51.960713 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1040/2058] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:49:51.965757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686ccc6-6c83-44ca-b72a-d8ca4dec2dad None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:51.975194 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686ccc6-6c83-44ca-b72a-d8ca4dec2dad None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:51.980696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030a279-4d7d-4e7e-8063-ea429b5c9eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:51.981371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030a279-4d7d-4e7e-8063-ea429b5c9eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:51.981658 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030a279-4d7d-4e7e-8063-ea429b5c9eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:51.981933 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030a279-4d7d-4e7e-8063-ea429b5c9eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:51.982601 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030a279-4d7d-4e7e-8063-ea429b5c9eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:51.984538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030a279-4d7d-4e7e-8063-ea429b5c9eb8 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:52.081256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1019/2059] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:51 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:52.086920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1d5e2-8e76-4637-91bd-34060947f0cd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:52.098093 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1d5e2-8e76-4637-91bd-34060947f0cd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:52.121516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:52.175362 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec51c8-2df4-4e34-be7e-48eeb3f02a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:52.176078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec51c8-2df4-4e34-be7e-48eeb3f02a68 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:52.176452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec51c8-2df4-4e34-be7e-48eeb3f02a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:52.176804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec51c8-2df4-4e34-be7e-48eeb3f02a68 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:52.177522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec51c8-2df4-4e34-be7e-48eeb3f02a68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:52.179556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec51c8-2df4-4e34-be7e-48eeb3f02a68 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:52.281254 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1041/2060] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:52 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:52.282616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030a279-4d7d-4e7e-8063-ea429b5c9eb8 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:52.293830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030a279-4d7d-4e7e-8063-ea429b5c9eb8 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:52.298921 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a841b-ebe1-45a7-82dc-2d249a883e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:52.299672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a841b-ebe1-45a7-82dc-2d249a883e23 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:52.300024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a841b-ebe1-45a7-82dc-2d249a883e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:52.300367 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a841b-ebe1-45a7-82dc-2d249a883e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:52.301132 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a841b-ebe1-45a7-82dc-2d249a883e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:52.302804 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a841b-ebe1-45a7-82dc-2d249a883e23 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:52.316458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1020/2061] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:52.319245 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec51c8-2df4-4e34-be7e-48eeb3f02a68 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:52.331276 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec51c8-2df4-4e34-be7e-48eeb3f02a68 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:52.348705 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d0c22-6e94-42c9-ab61-1ea2d8d0744a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:52.351271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d0c22-6e94-42c9-ab61-1ea2d8d0744a admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:52.351271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d0c22-6e94-42c9-ab61-1ea2d8d0744a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:52.351271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d0c22-6e94-42c9-ab61-1ea2d8d0744a admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:52.351271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d0c22-6e94-42c9-ab61-1ea2d8d0744a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:52.352362 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d0c22-6e94-42c9-ab61-1ea2d8d0744a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:52.362920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1042/2062] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:52.367079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a841b-ebe1-45a7-82dc-2d249a883e23 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:52.379133 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a841b-ebe1-45a7-82dc-2d249a883e23 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:52.383837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3b457-5e3f-443c-902d-6cb70c22c124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:52.384591 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3b457-5e3f-443c-902d-6cb70c22c124 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:52.384888 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3b457-5e3f-443c-902d-6cb70c22c124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:52.385148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3b457-5e3f-443c-902d-6cb70c22c124 admin admin] PATH_INFO: `/v3/projects/f221d65df9894bed8ac956852284fe85/users/73af9b2fa3044974bc4bc9dd9421a458/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:52.401789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3b457-5e3f-443c-902d-6cb70c22c124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f221d65df9894bed8ac956852284fe85, user_id=73af9b2fa3044974bc4bc9dd9421a458, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:52.404767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3b457-5e3f-443c-902d-6cb70c22c124 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:52.486137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1021/2063] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:52 2022] PUT /identity/v3/projects/f221d65df9894bed8ac956852284fe85/users/73af9b2fa3044974bc4bc9dd9421a458/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:52.498183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59d339-66d3-47ba-b662-e662809ecebb None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:52.498645 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59d339-66d3-47ba-b662-e662809ecebb None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:52.499053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59d339-66d3-47ba-b662-e662809ecebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:52.531828 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-2e59d339-66d3-47ba-b662-e662809ecebb None None] MFA Rules not processed for user `73af9b2fa3044974bc4bc9dd9421a458`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:52.533658 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2e59d339-66d3-47ba-b662-e662809ecebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:52.640065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257efdf-e3a4-4bbc-8ae6-ae250d3364d2 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:52.640466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257efdf-e3a4-4bbc-8ae6-ae250d3364d2 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:52.640822 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257efdf-e3a4-4bbc-8ae6-ae250d3364d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:52.673601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1043/2064] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:52.677945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59d339-66d3-47ba-b662-e662809ecebb None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:52.679859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-c257efdf-e3a4-4bbc-8ae6-ae250d3364d2 None None] MFA Rules not processed for user `55975de527254dca90fc4f30bc50e393`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:52.687614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c257efdf-e3a4-4bbc-8ae6-ae250d3364d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:52.703244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:52.762099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1022/2065] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:52.767524 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59d339-66d3-47ba-b662-e662809ecebb None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:52.772299 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12212426-24e3-4512-a50f-81cb153e25cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:52.772976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12212426-24e3-4512-a50f-81cb153e25cd admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:52.773498 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12212426-24e3-4512-a50f-81cb153e25cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:52.773781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12212426-24e3-4512-a50f-81cb153e25cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:52.774584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12212426-24e3-4512-a50f-81cb153e25cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:52.776571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12212426-24e3-4512-a50f-81cb153e25cd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:52.886498 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1044/2066] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:52 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:52.890609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257efdf-e3a4-4bbc-8ae6-ae250d3364d2 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:52.902640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257efdf-e3a4-4bbc-8ae6-ae250d3364d2 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:52.971100 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e44ced-7334-48af-894f-1a07521c210c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:52.971819 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e44ced-7334-48af-894f-1a07521c210c admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:52.972174 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e44ced-7334-48af-894f-1a07521c210c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:52.972540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e44ced-7334-48af-894f-1a07521c210c admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:52.973251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e44ced-7334-48af-894f-1a07521c210c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:52.975185 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e44ced-7334-48af-894f-1a07521c210c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:53.054907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1023/2067] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:52 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:53.056428 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12212426-24e3-4512-a50f-81cb153e25cd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:53.067382 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12212426-24e3-4512-a50f-81cb153e25cd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:53.072306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294a74a-38f4-4c8d-87d6-53798ee7dd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:53.073078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294a74a-38f4-4c8d-87d6-53798ee7dd2e admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:53.073391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294a74a-38f4-4c8d-87d6-53798ee7dd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:53.073668 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294a74a-38f4-4c8d-87d6-53798ee7dd2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:53.074323 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294a74a-38f4-4c8d-87d6-53798ee7dd2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:53.075907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294a74a-38f4-4c8d-87d6-53798ee7dd2e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:53.090090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1045/2068] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:53.091831 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e44ced-7334-48af-894f-1a07521c210c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:53.102228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e44ced-7334-48af-894f-1a07521c210c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:53.107514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c76568-72cc-4ec6-b867-e5d2fd345c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:53.108209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76568-72cc-4ec6-b867-e5d2fd345c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:53.108663 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76568-72cc-4ec6-b867-e5d2fd345c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:53.108949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76568-72cc-4ec6-b867-e5d2fd345c54 admin admin] PATH_INFO: `/v3/projects/1bf7bc3a9954448ca29f741e9e5bb030/users/1907888b3644404a894a53c7dfec84ef/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:53.124901 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c76568-72cc-4ec6-b867-e5d2fd345c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf7bc3a9954448ca29f741e9e5bb030, user_id=1907888b3644404a894a53c7dfec84ef, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:53.127692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c76568-72cc-4ec6-b867-e5d2fd345c54 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:53.187578 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1024/2069] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:53 2022] PUT /identity/v3/projects/1bf7bc3a9954448ca29f741e9e5bb030/users/1907888b3644404a894a53c7dfec84ef/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:53.189149 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294a74a-38f4-4c8d-87d6-53798ee7dd2e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:53.201120 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294a74a-38f4-4c8d-87d6-53798ee7dd2e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:53.205612 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:53.268831 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f51c34-2eac-434c-a10e-da1ca4eedf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:53.269455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f51c34-2eac-434c-a10e-da1ca4eedf76 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:53.269741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f51c34-2eac-434c-a10e-da1ca4eedf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:53.270019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f51c34-2eac-434c-a10e-da1ca4eedf76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:53.270672 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f51c34-2eac-434c-a10e-da1ca4eedf76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:53.272425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f51c34-2eac-434c-a10e-da1ca4eedf76 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:53.286889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1046/2070] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:53.287895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c76568-72cc-4ec6-b867-e5d2fd345c54 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:53.299146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c76568-72cc-4ec6-b867-e5d2fd345c54 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:53.304048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc4ea9-8cbb-4e55-89a6-78325083efd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:53.304748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc4ea9-8cbb-4e55-89a6-78325083efd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:53.305145 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc4ea9-8cbb-4e55-89a6-78325083efd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:53.305421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc4ea9-8cbb-4e55-89a6-78325083efd6 admin admin] PATH_INFO: `/v3/projects/1bf7bc3a9954448ca29f741e9e5bb030/users/1907888b3644404a894a53c7dfec84ef/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:53.308259 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc4ea9-8cbb-4e55-89a6-78325083efd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf7bc3a9954448ca29f741e9e5bb030, user_id=1907888b3644404a894a53c7dfec84ef, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:53.311992 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc4ea9-8cbb-4e55-89a6-78325083efd6 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:53.322192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1025/2071] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:53 2022] PUT /identity/v3/projects/1bf7bc3a9954448ca29f741e9e5bb030/users/1907888b3644404a894a53c7dfec84ef/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:53.332485 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7455505-c076-45a3-8c03-002f85242f36 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:53.332812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7455505-c076-45a3-8c03-002f85242f36 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:53.333090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7455505-c076-45a3-8c03-002f85242f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:53.363027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c7455505-c076-45a3-8c03-002f85242f36 None None] MFA Rules not processed for user `1907888b3644404a894a53c7dfec84ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:53.364238 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c7455505-c076-45a3-8c03-002f85242f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:53.435927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1047/2072] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:53.446645 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc4ea9-8cbb-4e55-89a6-78325083efd6 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:53.448969 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-f2fc4ea9-8cbb-4e55-89a6-78325083efd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:53.506953 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc4ea9-8cbb-4e55-89a6-78325083efd6 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:53.511840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8e877-9560-480e-8d8d-de2f416cacd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:53.512480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8e877-9560-480e-8d8d-de2f416cacd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:53.512760 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8e877-9560-480e-8d8d-de2f416cacd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:53.513043 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8e877-9560-480e-8d8d-de2f416cacd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:53.522217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8e877-9560-480e-8d8d-de2f416cacd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:53.523634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8e877-9560-480e-8d8d-de2f416cacd8 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:53.534417 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1026/2073] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:53.624692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f679b-fff7-41c3-950a-563d76657917 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:53.628488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f679b-fff7-41c3-950a-563d76657917 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:53.628488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f679b-fff7-41c3-950a-563d76657917 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:53.628488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1048/2074] 10.222.0.70 () {54 vars in 921 bytes} [Wed Aug 24 16:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:49:53.644107 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83b79f-f8a3-44a2-abe4-ea60e9f18a1b None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:53.644509 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83b79f-f8a3-44a2-abe4-ea60e9f18a1b None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:53.644851 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83b79f-f8a3-44a2-abe4-ea60e9f18a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:53.688084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-fa83b79f-f8a3-44a2-abe4-ea60e9f18a1b None None] MFA Rules not processed for user `fc4c43c85a5d49979ba803fdb1041c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:53.689408 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-fa83b79f-f8a3-44a2-abe4-ea60e9f18a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:53.724723 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:53.796363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1027/2075] 10.222.0.70 () {58 vars in 1018 bytes} [Wed Aug 24 16:49:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:53.823893 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f679b-fff7-41c3-950a-563d76657917 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:53.826356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0a5f679b-fff7-41c3-950a-563d76657917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:53.830444 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83b79f-f8a3-44a2-abe4-ea60e9f18a1b None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:53.832887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-fa83b79f-f8a3-44a2-abe4-ea60e9f18a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:53.855050 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83b79f-f8a3-44a2-abe4-ea60e9f18a1b None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:53.860643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb685462-6876-4b07-b4e5-0244bf891c29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:53.861429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb685462-6876-4b07-b4e5-0244bf891c29 service placement] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:53.861812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb685462-6876-4b07-b4e5-0244bf891c29 service placement] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:53.862197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb685462-6876-4b07-b4e5-0244bf891c29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:53.872638 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb685462-6876-4b07-b4e5-0244bf891c29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:53.874649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb685462-6876-4b07-b4e5-0244bf891c29 service placement] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:53.895382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1028/2076] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:53.935840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f679b-fff7-41c3-950a-563d76657917 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:53.940435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ef5f8-0b5c-4ea2-9873-a23ea128d7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:53.941078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ef5f8-0b5c-4ea2-9873-a23ea128d7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:53.941359 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ef5f8-0b5c-4ea2-9873-a23ea128d7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:53.941639 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ef5f8-0b5c-4ea2-9873-a23ea128d7c6 admin admin] PATH_INFO: `/v3/users/e2257adc34324452a325bb7786d02515` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:53.942783 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ef5f8-0b5c-4ea2-9873-a23ea128d7c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2257adc34324452a325bb7786d02515)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:53.944862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ef5f8-0b5c-4ea2-9873-a23ea128d7c6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:54.097475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:54.111446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e70ef5f8-0b5c-4ea2-9873-a23ea128d7c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2257adc34324452a325bb7786d02515'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:54.112119 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e70ef5f8-0b5c-4ea2-9873-a23ea128d7c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2257adc34324452a325bb7786d02515'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:54.112833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e70ef5f8-0b5c-4ea2-9873-a23ea128d7c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2257adc34324452a325bb7786d02515'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:54.190049 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e70ef5f8-0b5c-4ea2-9873-a23ea128d7c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2257adc34324452a325bb7786d02515'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:54.197809 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e70ef5f8-0b5c-4ea2-9873-a23ea128d7c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2257adc34324452a325bb7786d02515'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:54.202805 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb685462-6876-4b07-b4e5-0244bf891c29 service placement] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:54.213119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb685462-6876-4b07-b4e5-0244bf891c29 service placement] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:54.215897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1049/2077] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:53 2022] DELETE /identity/v3/users/e2257adc34324452a325bb7786d02515 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:54.217641 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09099a0a-829b-45c8-8be2-cafebeda34c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:54.218262 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09099a0a-829b-45c8-8be2-cafebeda34c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:54.218546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09099a0a-829b-45c8-8be2-cafebeda34c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:54.218823 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09099a0a-829b-45c8-8be2-cafebeda34c6 admin admin] PATH_INFO: `/v3/users/73af9b2fa3044974bc4bc9dd9421a458` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:54.220151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09099a0a-829b-45c8-8be2-cafebeda34c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73af9b2fa3044974bc4bc9dd9421a458)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:54.222109 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09099a0a-829b-45c8-8be2-cafebeda34c6 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:54.413133 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-09099a0a-829b-45c8-8be2-cafebeda34c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73af9b2fa3044974bc4bc9dd9421a458'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:54.413822 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-09099a0a-829b-45c8-8be2-cafebeda34c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73af9b2fa3044974bc4bc9dd9421a458'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:54.414539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-09099a0a-829b-45c8-8be2-cafebeda34c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73af9b2fa3044974bc4bc9dd9421a458'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:54.422567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-09099a0a-829b-45c8-8be2-cafebeda34c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73af9b2fa3044974bc4bc9dd9421a458'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:54.454903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ef5f8-0b5c-4ea2-9873-a23ea128d7c6 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:54.456317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e70ef5f8-0b5c-4ea2-9873-a23ea128d7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:54.501925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-09099a0a-829b-45c8-8be2-cafebeda34c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73af9b2fa3044974bc4bc9dd9421a458'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:54.520171 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1029/2078] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:54 2022] DELETE /identity/v3/users/73af9b2fa3044974bc4bc9dd9421a458 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:54.524562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9913a-62b9-4492-9e25-d9d093a6010a None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:54.524807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ef5f8-0b5c-4ea2-9873-a23ea128d7c6 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:54.524924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9913a-62b9-4492-9e25-d9d093a6010a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:54.525186 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9913a-62b9-4492-9e25-d9d093a6010a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:54.551178 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-06a9913a-62b9-4492-9e25-d9d093a6010a None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:54.552433 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-06a9913a-62b9-4492-9e25-d9d093a6010a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:54.586711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0f3b9-9258-42c0-8130-5513686613bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:54.587407 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0f3b9-9258-42c0-8130-5513686613bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:54.587824 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0f3b9-9258-42c0-8130-5513686613bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:54.588155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0f3b9-9258-42c0-8130-5513686613bf admin admin] PATH_INFO: `/v3/projects/3b5a3a82d90e4811a3037a8cda927913` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:54.590498 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0f3b9-9258-42c0-8130-5513686613bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b5a3a82d90e4811a3037a8cda927913)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:54.593981 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0f3b9-9258-42c0-8130-5513686613bf admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:54.626912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1030/2079] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:54.636529 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9913a-62b9-4492-9e25-d9d093a6010a None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:54.637928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-06a9913a-62b9-4492-9e25-d9d093a6010a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:54.652381 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:54.695151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9913a-62b9-4492-9e25-d9d093a6010a None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:54.698872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7687fd-9ae4-4468-be8d-588356da98c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:54.699480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7687fd-9ae4-4468-be8d-588356da98c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:54.699763 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7687fd-9ae4-4468-be8d-588356da98c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:54.700039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7687fd-9ae4-4468-be8d-588356da98c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:54.709101 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7687fd-9ae4-4468-be8d-588356da98c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:54.710523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7687fd-9ae4-4468-be8d-588356da98c8 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:54.723922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1031/2080] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:49:54.734235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-acf0f3b9-9258-42c0-8130-5513686613bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b5a3a82d90e4811a3037a8cda927913', 'request_id': 'req-acf0f3b9-9258-42c0-8130-5513686613bf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:54.742066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-acf0f3b9-9258-42c0-8130-5513686613bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b5a3a82d90e4811a3037a8cda927913', 'request_id': 'req-acf0f3b9-9258-42c0-8130-5513686613bf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:54.746648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:54.779152 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-acf0f3b9-9258-42c0-8130-5513686613bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b5a3a82d90e4811a3037a8cda927913', 'request_id': 'req-acf0f3b9-9258-42c0-8130-5513686613bf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:54.850130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-acf0f3b9-9258-42c0-8130-5513686613bf admin admin] The token cache is being invalidate because project 3b5a3a82d90e4811a3037a8cda927913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:54.850451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-acf0f3b9-9258-42c0-8130-5513686613bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:54.851159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-acf0f3b9-9258-42c0-8130-5513686613bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:54.852830 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1050/2081] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:54 2022] DELETE /identity/v3/projects/3b5a3a82d90e4811a3037a8cda927913 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:54.863850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1a241-9044-46f5-affa-19600d46f819 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:54.864141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1a241-9044-46f5-affa-19600d46f819 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:54.864458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1a241-9044-46f5-affa-19600d46f819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:54.891477 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-dee1a241-9044-46f5-affa-19600d46f819 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:54.892693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-dee1a241-9044-46f5-affa-19600d46f819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:54.958743 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1032/2082] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:54.967665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c806c-3e8b-42d0-a776-4245daa3cb1b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:54.968505 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c806c-3e8b-42d0-a776-4245daa3cb1b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:54.968924 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c806c-3e8b-42d0-a776-4245daa3cb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:54.994889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-af6c806c-3e8b-42d0-a776-4245daa3cb1b None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:54.994889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-af6c806c-3e8b-42d0-a776-4245daa3cb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:55.017913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1051/2083] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:55.027383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1a241-9044-46f5-affa-19600d46f819 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:55.037015 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1a241-9044-46f5-affa-19600d46f819 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:55.041888 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d0991-5614-48cd-b7b1-452e4ecd67f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:55.042491 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d0991-5614-48cd-b7b1-452e4ecd67f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:55.050911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d0991-5614-48cd-b7b1-452e4ecd67f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:55.050911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d0991-5614-48cd-b7b1-452e4ecd67f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:55.050911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d0991-5614-48cd-b7b1-452e4ecd67f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:49:55.050911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d0991-5614-48cd-b7b1-452e4ecd67f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:55.050911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d0991-5614-48cd-b7b1-452e4ecd67f4 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:55.061573 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1033/2084] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:49:55.062933 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c806c-3e8b-42d0-a776-4245daa3cb1b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:55.073456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c806c-3e8b-42d0-a776-4245daa3cb1b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:55.078302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f27f51-70e1-4fc6-ba2b-e2aded9d564b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:55.078937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f27f51-70e1-4fc6-ba2b-e2aded9d564b admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:55.079214 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f27f51-70e1-4fc6-ba2b-e2aded9d564b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:55.079515 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f27f51-70e1-4fc6-ba2b-e2aded9d564b admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:55.080156 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f27f51-70e1-4fc6-ba2b-e2aded9d564b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:55.082068 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f27f51-70e1-4fc6-ba2b-e2aded9d564b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:55.172610 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1052/2085] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:55 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:55.178133 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d0991-5614-48cd-b7b1-452e4ecd67f4 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:55.185186 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d0991-5614-48cd-b7b1-452e4ecd67f4 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:55.209176 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f27f51-70e1-4fc6-ba2b-e2aded9d564b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:55.211004 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-42f27f51-70e1-4fc6-ba2b-e2aded9d564b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:55.261748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3bffd-ec17-4001-a8d2-2b8972fdef45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:55.262359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3bffd-ec17-4001-a8d2-2b8972fdef45 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:55.262634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3bffd-ec17-4001-a8d2-2b8972fdef45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:55.267662 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3bffd-ec17-4001-a8d2-2b8972fdef45 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:55.267662 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3bffd-ec17-4001-a8d2-2b8972fdef45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:55.267662 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3bffd-ec17-4001-a8d2-2b8972fdef45 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:55.298016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f27f51-70e1-4fc6-ba2b-e2aded9d564b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:55.303042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5e0a8-84ae-436d-a271-0f09446f9e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:55.303762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5e0a8-84ae-436d-a271-0f09446f9e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:55.304172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5e0a8-84ae-436d-a271-0f09446f9e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:55.304364 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5e0a8-84ae-436d-a271-0f09446f9e58 admin admin] PATH_INFO: `/v3/projects/f221d65df9894bed8ac956852284fe85` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:55.305621 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5e0a8-84ae-436d-a271-0f09446f9e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f221d65df9894bed8ac956852284fe85)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:55.308073 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5e0a8-84ae-436d-a271-0f09446f9e58 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:55.382548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1034/2086] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:55 2022] POST /identity/v3/users => generated 285 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:55.383637 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3bffd-ec17-4001-a8d2-2b8972fdef45 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:55.394003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3bffd-ec17-4001-a8d2-2b8972fdef45 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:55.398574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb5b6a-9092-415f-b20d-a7840f567610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:55.399156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb5b6a-9092-415f-b20d-a7840f567610 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:55.399432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb5b6a-9092-415f-b20d-a7840f567610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:55.399701 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb5b6a-9092-415f-b20d-a7840f567610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:55.400331 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb5b6a-9092-415f-b20d-a7840f567610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:55.402055 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb5b6a-9092-415f-b20d-a7840f567610 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:55.421161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1035/2087] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:55.423731 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb5b6a-9092-415f-b20d-a7840f567610 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:55.433931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb5b6a-9092-415f-b20d-a7840f567610 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:55.438875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c5082-f06d-4edd-95c7-cbdf72d508eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:55.439468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c5082-f06d-4edd-95c7-cbdf72d508eb admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:55.439754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c5082-f06d-4edd-95c7-cbdf72d508eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:55.440029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c5082-f06d-4edd-95c7-cbdf72d508eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:55.440696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1c5082-f06d-4edd-95c7-cbdf72d508eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:55.442308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1c5082-f06d-4edd-95c7-cbdf72d508eb admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:55.456075 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1036/2088] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:49:55.457120 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c5082-f06d-4edd-95c7-cbdf72d508eb admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:55.457633 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-eab5e0a8-84ae-436d-a271-0f09446f9e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f221d65df9894bed8ac956852284fe85', 'request_id': 'req-eab5e0a8-84ae-436d-a271-0f09446f9e58'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:55.464219 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-eab5e0a8-84ae-436d-a271-0f09446f9e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f221d65df9894bed8ac956852284fe85', 'request_id': 'req-eab5e0a8-84ae-436d-a271-0f09446f9e58'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:55.466758 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c5082-f06d-4edd-95c7-cbdf72d508eb admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:55.470128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-eab5e0a8-84ae-436d-a271-0f09446f9e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f221d65df9894bed8ac956852284fe85', 'request_id': 'req-eab5e0a8-84ae-436d-a271-0f09446f9e58'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:55.474979 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:55.525322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12028eb-95ad-4181-8fea-5024816492b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:55.526662 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12028eb-95ad-4181-8fea-5024816492b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:55.526799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12028eb-95ad-4181-8fea-5024816492b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:55.527041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12028eb-95ad-4181-8fea-5024816492b0 admin admin] PATH_INFO: `/v3/projects/cb469a5b5781412887cd32a7dc805260/users/d04e14e4e8e64a93b53d3fb84d5a33a4/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:55.542554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12028eb-95ad-4181-8fea-5024816492b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb469a5b5781412887cd32a7dc805260, user_id=d04e14e4e8e64a93b53d3fb84d5a33a4, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:55.547574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12028eb-95ad-4181-8fea-5024816492b0 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:55.570011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-eab5e0a8-84ae-436d-a271-0f09446f9e58 admin admin] The token cache is being invalidate because project f221d65df9894bed8ac956852284fe85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:55.570303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-eab5e0a8-84ae-436d-a271-0f09446f9e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:55.570958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-eab5e0a8-84ae-436d-a271-0f09446f9e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:55.572626 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1053/2089] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:55 2022] DELETE /identity/v3/projects/f221d65df9894bed8ac956852284fe85 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:55.576416 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5e0a8-84ae-436d-a271-0f09446f9e58 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:55.577758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-eab5e0a8-84ae-436d-a271-0f09446f9e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:55.590783 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5e0a8-84ae-436d-a271-0f09446f9e58 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:55.596011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a25ba9-3834-46cc-8b32-f111e2a162e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:55.596660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a25ba9-3834-46cc-8b32-f111e2a162e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:55.599752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a25ba9-3834-46cc-8b32-f111e2a162e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:55.599752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a25ba9-3834-46cc-8b32-f111e2a162e1 admin admin] PATH_INFO: `/v3/users/1907888b3644404a894a53c7dfec84ef` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:55.599752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a25ba9-3834-46cc-8b32-f111e2a162e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1907888b3644404a894a53c7dfec84ef)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:55.600575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a25ba9-3834-46cc-8b32-f111e2a162e1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:55.659785 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1037/2090] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:55 2022] PUT /identity/v3/projects/cb469a5b5781412887cd32a7dc805260/users/d04e14e4e8e64a93b53d3fb84d5a33a4/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:55.669819 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8043a32-3afa-434b-897c-032fd50b264d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:55.670110 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8043a32-3afa-434b-897c-032fd50b264d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:55.670389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8043a32-3afa-434b-897c-032fd50b264d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:55.697439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-a8043a32-3afa-434b-897c-032fd50b264d None None] MFA Rules not processed for user `d04e14e4e8e64a93b53d3fb84d5a33a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:55.698383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a8043a32-3afa-434b-897c-032fd50b264d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:55.761705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1038/2091] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:55.763028 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:55.766230 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8043a32-3afa-434b-897c-032fd50b264d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:55.767529 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a8043a32-3afa-434b-897c-032fd50b264d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:55.768297 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a9a25ba9-3834-46cc-8b32-f111e2a162e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1907888b3644404a894a53c7dfec84ef'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:55.768997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a9a25ba9-3834-46cc-8b32-f111e2a162e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1907888b3644404a894a53c7dfec84ef'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:55.769745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a9a25ba9-3834-46cc-8b32-f111e2a162e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1907888b3644404a894a53c7dfec84ef'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:55.834663 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8043a32-3afa-434b-897c-032fd50b264d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:55.836171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a9a25ba9-3834-46cc-8b32-f111e2a162e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1907888b3644404a894a53c7dfec84ef'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:55.839998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f93157-93ab-434a-b9d7-7cd5ff1d036a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:55.840669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f93157-93ab-434a-b9d7-7cd5ff1d036a admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:55.840951 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f93157-93ab-434a-b9d7-7cd5ff1d036a admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:55.841226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f93157-93ab-434a-b9d7-7cd5ff1d036a admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:55.841854 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f93157-93ab-434a-b9d7-7cd5ff1d036a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:55.844410 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f93157-93ab-434a-b9d7-7cd5ff1d036a admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:55.844527 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a9a25ba9-3834-46cc-8b32-f111e2a162e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1907888b3644404a894a53c7dfec84ef'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:55.862853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1054/2092] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:55 2022] DELETE /identity/v3/users/1907888b3644404a894a53c7dfec84ef => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:55.940808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1039/2093] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:55 2022] POST /identity/v3/projects => generated 367 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:55.942019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a25ba9-3834-46cc-8b32-f111e2a162e1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:55.943584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a9a25ba9-3834-46cc-8b32-f111e2a162e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:56.029261 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a25ba9-3834-46cc-8b32-f111e2a162e1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:56.098813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf8901-972b-40d9-9af6-9a70d9598709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:56.099453 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf8901-972b-40d9-9af6-9a70d9598709 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:56.099739 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf8901-972b-40d9-9af6-9a70d9598709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:56.100016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf8901-972b-40d9-9af6-9a70d9598709 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:56.101482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf8901-972b-40d9-9af6-9a70d9598709 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:56.103588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf8901-972b-40d9-9af6-9a70d9598709 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:56.227883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1055/2094] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:55 2022] POST /identity/v3/users => generated 290 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:56.231708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f93157-93ab-434a-b9d7-7cd5ff1d036a admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:56.242793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f93157-93ab-434a-b9d7-7cd5ff1d036a admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:56.248391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930ce2c-0266-4149-9558-01d59908dfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:56.250097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930ce2c-0266-4149-9558-01d59908dfbf admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:56.250382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930ce2c-0266-4149-9558-01d59908dfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:56.250672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930ce2c-0266-4149-9558-01d59908dfbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:56.251302 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930ce2c-0266-4149-9558-01d59908dfbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:56.253078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930ce2c-0266-4149-9558-01d59908dfbf admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:56.255775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:56.269971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1040/2095] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:49:56.274246 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf8901-972b-40d9-9af6-9a70d9598709 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:56.285084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf8901-972b-40d9-9af6-9a70d9598709 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:56.299422 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1edeae-f2b9-4bd5-a8e9-0e74c2d693f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:56.300113 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1edeae-f2b9-4bd5-a8e9-0e74c2d693f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:56.300392 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1edeae-f2b9-4bd5-a8e9-0e74c2d693f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:56.300726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1edeae-f2b9-4bd5-a8e9-0e74c2d693f4 admin admin] PATH_INFO: `/v3/projects/07105c938ec14a66bd933f8c1efada01/users/919a84a62d5541e891b0deabef96cde2/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:56.318382 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1edeae-f2b9-4bd5-a8e9-0e74c2d693f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07105c938ec14a66bd933f8c1efada01, user_id=919a84a62d5541e891b0deabef96cde2, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:56.321343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1edeae-f2b9-4bd5-a8e9-0e74c2d693f4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:56.402854 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1056/2096] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:56 2022] PUT /identity/v3/projects/07105c938ec14a66bd933f8c1efada01/users/919a84a62d5541e891b0deabef96cde2/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:56.405371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930ce2c-0266-4149-9558-01d59908dfbf admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:56.406585 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1edeae-f2b9-4bd5-a8e9-0e74c2d693f4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:56.407548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1930ce2c-0266-4149-9558-01d59908dfbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:56.421634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930ce2c-0266-4149-9558-01d59908dfbf admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:56.450555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:56.492878 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1edeae-f2b9-4bd5-a8e9-0e74c2d693f4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:56.499015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5915ab-62ed-4f78-8522-0e1ba391a4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:56.499941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5915ab-62ed-4f78-8522-0e1ba391a4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:56.499941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5915ab-62ed-4f78-8522-0e1ba391a4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:56.500316 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5915ab-62ed-4f78-8522-0e1ba391a4c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:56.501234 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5915ab-62ed-4f78-8522-0e1ba391a4c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:56.502122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afaebb7-7f0c-4dca-bb23-dd1e9d506d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:56.502814 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaebb7-7f0c-4dca-bb23-dd1e9d506d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:56.502933 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5915ab-62ed-4f78-8522-0e1ba391a4c7 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:56.503097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaebb7-7f0c-4dca-bb23-dd1e9d506d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:56.503365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaebb7-7f0c-4dca-bb23-dd1e9d506d29 admin admin] PATH_INFO: `/v3/projects/1bf7bc3a9954448ca29f741e9e5bb030` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:56.505038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afaebb7-7f0c-4dca-bb23-dd1e9d506d29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bf7bc3a9954448ca29f741e9e5bb030)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:56.507941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afaebb7-7f0c-4dca-bb23-dd1e9d506d29 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:56.517228 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1057/2097] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:56.518449 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5915ab-62ed-4f78-8522-0e1ba391a4c7 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:56.528336 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5915ab-62ed-4f78-8522-0e1ba391a4c7 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:56.533013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafb4d4-a2fb-4215-bfc1-1197c3735b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:56.533730 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafb4d4-a2fb-4215-bfc1-1197c3735b3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:56.534012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafb4d4-a2fb-4215-bfc1-1197c3735b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:56.534281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafb4d4-a2fb-4215-bfc1-1197c3735b3e admin admin] PATH_INFO: `/v3/projects/07105c938ec14a66bd933f8c1efada01/users/919a84a62d5541e891b0deabef96cde2/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:56.536433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafb4d4-a2fb-4215-bfc1-1197c3735b3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=07105c938ec14a66bd933f8c1efada01, user_id=919a84a62d5541e891b0deabef96cde2, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:56.539445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafb4d4-a2fb-4215-bfc1-1197c3735b3e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:56.550802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1058/2098] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:56 2022] PUT /identity/v3/projects/07105c938ec14a66bd933f8c1efada01/users/919a84a62d5541e891b0deabef96cde2/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:56.563337 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db5c36-a9bc-45f0-9ced-6fa027cd9370 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:56.563627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db5c36-a9bc-45f0-9ced-6fa027cd9370 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:56.564164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db5c36-a9bc-45f0-9ced-6fa027cd9370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:56.591877 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-38db5c36-a9bc-45f0-9ced-6fa027cd9370 None None] MFA Rules not processed for user `919a84a62d5541e891b0deabef96cde2`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:56.596541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-38db5c36-a9bc-45f0-9ced-6fa027cd9370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:56.638534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2afaebb7-7f0c-4dca-bb23-dd1e9d506d29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bf7bc3a9954448ca29f741e9e5bb030', 'request_id': 'req-2afaebb7-7f0c-4dca-bb23-dd1e9d506d29'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:56.646403 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2afaebb7-7f0c-4dca-bb23-dd1e9d506d29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bf7bc3a9954448ca29f741e9e5bb030', 'request_id': 'req-2afaebb7-7f0c-4dca-bb23-dd1e9d506d29'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:56.654488 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2afaebb7-7f0c-4dca-bb23-dd1e9d506d29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bf7bc3a9954448ca29f741e9e5bb030', 'request_id': 'req-2afaebb7-7f0c-4dca-bb23-dd1e9d506d29'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:56.667760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1059/2099] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:56.678314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db5c36-a9bc-45f0-9ced-6fa027cd9370 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:56.680120 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-38db5c36-a9bc-45f0-9ced-6fa027cd9370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:56.742849 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2afaebb7-7f0c-4dca-bb23-dd1e9d506d29 admin admin] The token cache is being invalidate because project 1bf7bc3a9954448ca29f741e9e5bb030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:56.743948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2afaebb7-7f0c-4dca-bb23-dd1e9d506d29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:56.745250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db5c36-a9bc-45f0-9ced-6fa027cd9370 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:56.745433 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2afaebb7-7f0c-4dca-bb23-dd1e9d506d29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:56.747031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1041/2100] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:56 2022] DELETE /identity/v3/projects/1bf7bc3a9954448ca29f741e9e5bb030 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:56.753091 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88558a7-b478-4ef3-b582-ec5642b78337 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:56.753386 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88558a7-b478-4ef3-b582-ec5642b78337 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:56.753659 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88558a7-b478-4ef3-b582-ec5642b78337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:56.774517 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-e88558a7-b478-4ef3-b582-ec5642b78337 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:56.775748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e88558a7-b478-4ef3-b582-ec5642b78337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:56.807032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977011b4-89ae-444c-93d9-57e1770008db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:56.807847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977011b4-89ae-444c-93d9-57e1770008db service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:56.808458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977011b4-89ae-444c-93d9-57e1770008db service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:56.808745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977011b4-89ae-444c-93d9-57e1770008db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:56.810601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-977011b4-89ae-444c-93d9-57e1770008db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:56.821080 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:56.864554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1042/2101] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:56.868842 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4550fa-4aa6-4e66-a6a9-f3a91946e4a4 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:56.869475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4550fa-4aa6-4e66-a6a9-f3a91946e4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:56.869907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4550fa-4aa6-4e66-a6a9-f3a91946e4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:56.891183 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-ae4550fa-4aa6-4e66-a6a9-f3a91946e4a4 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:56.892455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ae4550fa-4aa6-4e66-a6a9-f3a91946e4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:56.903692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1043/2102] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:56.904035 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977011b4-89ae-444c-93d9-57e1770008db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:56.905838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977011b4-89ae-444c-93d9-57e1770008db service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:56.907647 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4550fa-4aa6-4e66-a6a9-f3a91946e4a4 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:56.917782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1060/2103] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:49:56.923291 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4550fa-4aa6-4e66-a6a9-f3a91946e4a4 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:56.923832 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4c34e-61cd-420f-afd0-540268f8e6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:56.924646 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4c34e-61cd-420f-afd0-540268f8e6b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:56.924832 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4c34e-61cd-420f-afd0-540268f8e6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:56.925093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4c34e-61cd-420f-afd0-540268f8e6b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:56.925777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc4c34e-61cd-420f-afd0-540268f8e6b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:49:56.926083 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc4c34e-61cd-420f-afd0-540268f8e6b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:56.927712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc4c34e-61cd-420f-afd0-540268f8e6b3 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:56.940172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1044/2104] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:49:56.945144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977011b4-89ae-444c-93d9-57e1770008db service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:56.955327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977011b4-89ae-444c-93d9-57e1770008db service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:56.960061 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74542a28-a945-4638-9a7d-78bab49dabb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:56.960697 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74542a28-a945-4638-9a7d-78bab49dabb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:56.961034 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74542a28-a945-4638-9a7d-78bab49dabb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:56.961289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74542a28-a945-4638-9a7d-78bab49dabb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:56.961896 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74542a28-a945-4638-9a7d-78bab49dabb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:56.964440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74542a28-a945-4638-9a7d-78bab49dabb3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:57.082829 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1061/2105] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:56 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:57.084012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4c34e-61cd-420f-afd0-540268f8e6b3 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:57.094782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4c34e-61cd-420f-afd0-540268f8e6b3 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:57.100235 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:57.153182 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb905f-810a-472a-b024-77a5cb321cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:57.153791 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb905f-810a-472a-b024-77a5cb321cca admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:57.154067 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb905f-810a-472a-b024-77a5cb321cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:57.154336 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb905f-810a-472a-b024-77a5cb321cca admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:57.154957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb905f-810a-472a-b024-77a5cb321cca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:57.156918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb905f-810a-472a-b024-77a5cb321cca admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:57.295942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1045/2106] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:57 2022] POST /identity/v3/users => generated 287 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:57.297559 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74542a28-a945-4638-9a7d-78bab49dabb3 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:57.309429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74542a28-a945-4638-9a7d-78bab49dabb3 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:57.317884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf36ae-fd18-4b6d-bd67-5c934364e50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:57.317884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf36ae-fd18-4b6d-bd67-5c934364e50a admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:57.317884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf36ae-fd18-4b6d-bd67-5c934364e50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:57.317884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf36ae-fd18-4b6d-bd67-5c934364e50a admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:57.318698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf36ae-fd18-4b6d-bd67-5c934364e50a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:57.321008 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf36ae-fd18-4b6d-bd67-5c934364e50a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:57.336452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1062/2107] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:49:57.340845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb905f-810a-472a-b024-77a5cb321cca admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:57.355660 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb905f-810a-472a-b024-77a5cb321cca admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:57.361245 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ff8df-535c-4fce-8966-434926d482af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:57.361939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ff8df-535c-4fce-8966-434926d482af admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:57.362702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ff8df-535c-4fce-8966-434926d482af admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:57.362702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ff8df-535c-4fce-8966-434926d482af admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:57.364294 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ff8df-535c-4fce-8966-434926d482af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:57.365470 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ff8df-535c-4fce-8966-434926d482af admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:57.380566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1046/2108] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:57.384304 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf36ae-fd18-4b6d-bd67-5c934364e50a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:57.395736 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf36ae-fd18-4b6d-bd67-5c934364e50a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:57.405162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d370c6-3bb0-4917-b1de-0a38800b4db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:57.406248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d370c6-3bb0-4917-b1de-0a38800b4db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:57.406684 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d370c6-3bb0-4917-b1de-0a38800b4db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:57.407108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d370c6-3bb0-4917-b1de-0a38800b4db2 admin admin] PATH_INFO: `/v3/projects/48d5536c39ab4a13bb7d991b2acf08e0/users/6b6bbaa4db834a0996dc0307595c8a30/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:57.428272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d370c6-3bb0-4917-b1de-0a38800b4db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d5536c39ab4a13bb7d991b2acf08e0, user_id=6b6bbaa4db834a0996dc0307595c8a30, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:57.431308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d370c6-3bb0-4917-b1de-0a38800b4db2 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:57.540671 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1063/2109] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:57 2022] PUT /identity/v3/projects/48d5536c39ab4a13bb7d991b2acf08e0/users/6b6bbaa4db834a0996dc0307595c8a30/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:57.552971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afc887-c8be-41f3-820e-47400f630bdd None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:57.554212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afc887-c8be-41f3-820e-47400f630bdd None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:57.554503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afc887-c8be-41f3-820e-47400f630bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:57.591360 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-e7afc887-c8be-41f3-820e-47400f630bdd None None] MFA Rules not processed for user `6b6bbaa4db834a0996dc0307595c8a30`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:57.592600 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e7afc887-c8be-41f3-820e-47400f630bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:57.666627 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1047/2110] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:57.678937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d370c6-3bb0-4917-b1de-0a38800b4db2 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:57.680764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d9d370c6-3bb0-4917-b1de-0a38800b4db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:57.738132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d370c6-3bb0-4917-b1de-0a38800b4db2 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:57.741953 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5653c7-dd66-46e5-9f95-07b0cd502c9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:57.742557 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5653c7-dd66-46e5-9f95-07b0cd502c9d service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:57.742838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5653c7-dd66-46e5-9f95-07b0cd502c9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:57.743110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5653c7-dd66-46e5-9f95-07b0cd502c9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:57.752672 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5653c7-dd66-46e5-9f95-07b0cd502c9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:57.754092 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5653c7-dd66-46e5-9f95-07b0cd502c9d service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:57.765210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1064/2111] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:57.786680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afc887-c8be-41f3-820e-47400f630bdd None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:57.858779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afc887-c8be-41f3-820e-47400f630bdd None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:57.863429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4d103-7367-4140-b074-31cd3eb4df57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:57.864019 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4d103-7367-4140-b074-31cd3eb4df57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:57.864296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4d103-7367-4140-b074-31cd3eb4df57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:57.864594 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4d103-7367-4140-b074-31cd3eb4df57 admin admin] PATH_INFO: `/v3/users/6b6bbaa4db834a0996dc0307595c8a30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:57.865725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4d103-7367-4140-b074-31cd3eb4df57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b6bbaa4db834a0996dc0307595c8a30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:57.867652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4d103-7367-4140-b074-31cd3eb4df57 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:57.890715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:58.112892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-71f4d103-7367-4140-b074-31cd3eb4df57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b6bbaa4db834a0996dc0307595c8a30'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:58.113576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-71f4d103-7367-4140-b074-31cd3eb4df57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b6bbaa4db834a0996dc0307595c8a30'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:58.114192 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-71f4d103-7367-4140-b074-31cd3eb4df57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b6bbaa4db834a0996dc0307595c8a30'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:58.123386 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-71f4d103-7367-4140-b074-31cd3eb4df57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b6bbaa4db834a0996dc0307595c8a30'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:58.219423 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-71f4d103-7367-4140-b074-31cd3eb4df57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b6bbaa4db834a0996dc0307595c8a30'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:58.249268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1048/2112] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:57 2022] DELETE /identity/v3/users/6b6bbaa4db834a0996dc0307595c8a30 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:58.251174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2d193-3e96-4eb8-9875-1b22cb6c5320 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:58.251624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2d193-3e96-4eb8-9875-1b22cb6c5320 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:58.251758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2d193-3e96-4eb8-9875-1b22cb6c5320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:58.270811 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-52a2d193-3e96-4eb8-9875-1b22cb6c5320 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:58.272011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-52a2d193-3e96-4eb8-9875-1b22cb6c5320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:58.320205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:58.334348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1065/2113] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:58.347777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4d103-7367-4140-b074-31cd3eb4df57 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:58.349201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-71f4d103-7367-4140-b074-31cd3eb4df57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:58.453528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4d103-7367-4140-b074-31cd3eb4df57 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:58.460534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4343e36-a065-46d4-83ff-cab35da7ac24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:58.461162 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4343e36-a065-46d4-83ff-cab35da7ac24 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:58.461448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4343e36-a065-46d4-83ff-cab35da7ac24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:58.461722 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4343e36-a065-46d4-83ff-cab35da7ac24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:58.472721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4343e36-a065-46d4-83ff-cab35da7ac24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:58.474160 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4343e36-a065-46d4-83ff-cab35da7ac24 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:58.485078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1049/2114] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:59.346456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2d193-3e96-4eb8-9875-1b22cb6c5320 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:59.347933 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-52a2d193-3e96-4eb8-9875-1b22cb6c5320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:59.368930 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2d193-3e96-4eb8-9875-1b22cb6c5320 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:59.375797 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c3750-214a-41ad-984f-9a24dae0982d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:59.376503 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c3750-214a-41ad-984f-9a24dae0982d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:59.376791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c3750-214a-41ad-984f-9a24dae0982d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:59.377067 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c3750-214a-41ad-984f-9a24dae0982d admin admin] PATH_INFO: `/v3/projects/48d5536c39ab4a13bb7d991b2acf08e0` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:59.378530 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c3750-214a-41ad-984f-9a24dae0982d admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d5536c39ab4a13bb7d991b2acf08e0)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:59.380866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c3750-214a-41ad-984f-9a24dae0982d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:59.498737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4c7c3750-214a-41ad-984f-9a24dae0982d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d5536c39ab4a13bb7d991b2acf08e0', 'request_id': 'req-4c7c3750-214a-41ad-984f-9a24dae0982d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:59.506256 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4c7c3750-214a-41ad-984f-9a24dae0982d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d5536c39ab4a13bb7d991b2acf08e0', 'request_id': 'req-4c7c3750-214a-41ad-984f-9a24dae0982d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:59.512153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4c7c3750-214a-41ad-984f-9a24dae0982d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d5536c39ab4a13bb7d991b2acf08e0', 'request_id': 'req-4c7c3750-214a-41ad-984f-9a24dae0982d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:59.599144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4c7c3750-214a-41ad-984f-9a24dae0982d admin admin] The token cache is being invalidate because project 48d5536c39ab4a13bb7d991b2acf08e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:59.600816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4c7c3750-214a-41ad-984f-9a24dae0982d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:59.600816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4c7c3750-214a-41ad-984f-9a24dae0982d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:59.601885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1066/2115] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:59 2022] DELETE /identity/v3/projects/48d5536c39ab4a13bb7d991b2acf08e0 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:59.607359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49875959-5b37-408a-983e-d166f13846ff None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:59.607845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49875959-5b37-408a-983e-d166f13846ff None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:59.607939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49875959-5b37-408a-983e-d166f13846ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:59.630575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-49875959-5b37-408a-983e-d166f13846ff None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:59.631773 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-49875959-5b37-408a-983e-d166f13846ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:59.691458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1050/2116] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:59.700168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477eef4-4fff-482c-904a-c1c43dc91ed7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:59.700495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477eef4-4fff-482c-904a-c1c43dc91ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:59.700783 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477eef4-4fff-482c-904a-c1c43dc91ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:59.720170 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8477eef4-4fff-482c-904a-c1c43dc91ed7 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:59.721441 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8477eef4-4fff-482c-904a-c1c43dc91ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:59.733284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1067/2117] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:59.734390 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49875959-5b37-408a-983e-d166f13846ff None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:59.750567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49875959-5b37-408a-983e-d166f13846ff None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:59.750567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41701bf4-2d70-4a15-8371-8904cd625b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:59.750883 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41701bf4-2d70-4a15-8371-8904cd625b0b admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:59.751176 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41701bf4-2d70-4a15-8371-8904cd625b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:59.751453 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41701bf4-2d70-4a15-8371-8904cd625b0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:59.752110 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41701bf4-2d70-4a15-8371-8904cd625b0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:49:59.752441 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41701bf4-2d70-4a15-8371-8904cd625b0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:59.754037 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41701bf4-2d70-4a15-8371-8904cd625b0b admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:59.766475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1051/2118] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:49:59.777437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477eef4-4fff-482c-904a-c1c43dc91ed7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:59.786584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477eef4-4fff-482c-904a-c1c43dc91ed7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:59.791460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376f837-1122-411d-8e73-7476e00a0959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:59.792708 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376f837-1122-411d-8e73-7476e00a0959 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:59.792989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376f837-1122-411d-8e73-7476e00a0959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:59.793290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376f837-1122-411d-8e73-7476e00a0959 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:59.793931 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376f837-1122-411d-8e73-7476e00a0959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:59.795839 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376f837-1122-411d-8e73-7476e00a0959 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:59.894312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1068/2119] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:59 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:59.897182 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41701bf4-2d70-4a15-8371-8904cd625b0b admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:59.923447 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41701bf4-2d70-4a15-8371-8904cd625b0b admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:59.955273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:59.988715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf1acd-5f99-4a7f-9080-dded9b7a2e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:59.989351 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf1acd-5f99-4a7f-9080-dded9b7a2e64 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:59.989633 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf1acd-5f99-4a7f-9080-dded9b7a2e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:59.989905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf1acd-5f99-4a7f-9080-dded9b7a2e64 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:59.990530 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf1acd-5f99-4a7f-9080-dded9b7a2e64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:59.992521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf1acd-5f99-4a7f-9080-dded9b7a2e64 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:00.040641 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376f837-1122-411d-8e73-7476e00a0959 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:00.042922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c376f837-1122-411d-8e73-7476e00a0959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:00.059348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376f837-1122-411d-8e73-7476e00a0959 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:00.066124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933952c9-9f9e-49fe-8491-b75e89b0e5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:00.066890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933952c9-9f9e-49fe-8491-b75e89b0e5e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:00.067569 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933952c9-9f9e-49fe-8491-b75e89b0e5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:00.067855 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933952c9-9f9e-49fe-8491-b75e89b0e5e9 admin admin] PATH_INFO: `/v3/users/d04e14e4e8e64a93b53d3fb84d5a33a4` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:00.069120 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933952c9-9f9e-49fe-8491-b75e89b0e5e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d04e14e4e8e64a93b53d3fb84d5a33a4)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:00.071333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933952c9-9f9e-49fe-8491-b75e89b0e5e9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:00.083459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1052/2120] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:59 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:50:00.091703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf1acd-5f99-4a7f-9080-dded9b7a2e64 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:00.102176 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf1acd-5f99-4a7f-9080-dded9b7a2e64 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:00.114121 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242439c-1b5c-4ea2-9949-18890f1c6ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:00.114961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242439c-1b5c-4ea2-9949-18890f1c6ede admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:00.115240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242439c-1b5c-4ea2-9949-18890f1c6ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:00.115516 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242439c-1b5c-4ea2-9949-18890f1c6ede admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:00.116193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242439c-1b5c-4ea2-9949-18890f1c6ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:00.117845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242439c-1b5c-4ea2-9949-18890f1c6ede admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:00.129683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1053/2121] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:50:00.133932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242439c-1b5c-4ea2-9949-18890f1c6ede admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:00.143910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242439c-1b5c-4ea2-9949-18890f1c6ede admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:00.148604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d265b9-e903-464c-9de1-1c636d27b14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:00.149169 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d265b9-e903-464c-9de1-1c636d27b14b admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:00.149452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d265b9-e903-464c-9de1-1c636d27b14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:00.149727 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d265b9-e903-464c-9de1-1c636d27b14b admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:00.150346 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d265b9-e903-464c-9de1-1c636d27b14b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:00.152650 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d265b9-e903-464c-9de1-1c636d27b14b admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:00.162438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1054/2122] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:50:00.166205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d265b9-e903-464c-9de1-1c636d27b14b admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:00.175729 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d265b9-e903-464c-9de1-1c636d27b14b admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:00.180281 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731df751-ed3d-42f4-9235-50197151b518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:00.180977 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731df751-ed3d-42f4-9235-50197151b518 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:00.181261 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731df751-ed3d-42f4-9235-50197151b518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:00.181536 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731df751-ed3d-42f4-9235-50197151b518 admin admin] PATH_INFO: `/v3/projects/5e7c3f5fc1b3403da3f37a7e0afdc657/users/c480e0a5f049480b9f1ed0f877a25b83/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:00.195817 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731df751-ed3d-42f4-9235-50197151b518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e7c3f5fc1b3403da3f37a7e0afdc657, user_id=c480e0a5f049480b9f1ed0f877a25b83, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:00.199122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731df751-ed3d-42f4-9235-50197151b518 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:00.277374 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1055/2123] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:00 2022] PUT /identity/v3/projects/5e7c3f5fc1b3403da3f37a7e0afdc657/users/c480e0a5f049480b9f1ed0f877a25b83/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:50:00.285795 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-933952c9-9f9e-49fe-8491-b75e89b0e5e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd04e14e4e8e64a93b53d3fb84d5a33a4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:00.286458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-933952c9-9f9e-49fe-8491-b75e89b0e5e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd04e14e4e8e64a93b53d3fb84d5a33a4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:00.287071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-933952c9-9f9e-49fe-8491-b75e89b0e5e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd04e14e4e8e64a93b53d3fb84d5a33a4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:00.287388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f13c7-b348-4249-89e8-6199dbe7b49c None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:00.287679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f13c7-b348-4249-89e8-6199dbe7b49c None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:00.287957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f13c7-b348-4249-89e8-6199dbe7b49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:00.314867 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-0e2f13c7-b348-4249-89e8-6199dbe7b49c None None] MFA Rules not processed for user `c480e0a5f049480b9f1ed0f877a25b83`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:00.316064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-0e2f13c7-b348-4249-89e8-6199dbe7b49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:00.378708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1056/2124] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:00.383168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f13c7-b348-4249-89e8-6199dbe7b49c None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:00.384506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-0e2f13c7-b348-4249-89e8-6199dbe7b49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:00.384774 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-933952c9-9f9e-49fe-8491-b75e89b0e5e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd04e14e4e8e64a93b53d3fb84d5a33a4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:00.394574 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-933952c9-9f9e-49fe-8491-b75e89b0e5e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd04e14e4e8e64a93b53d3fb84d5a33a4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:00.414690 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1069/2125] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:00 2022] DELETE /identity/v3/users/d04e14e4e8e64a93b53d3fb84d5a33a4 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:00.419447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831233e-1a26-4bf1-bb4f-a156191a8ff1 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:00.419798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831233e-1a26-4bf1-bb4f-a156191a8ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:00.420110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831233e-1a26-4bf1-bb4f-a156191a8ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:00.439314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4831233e-1a26-4bf1-bb4f-a156191a8ff1 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:00.440592 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4831233e-1a26-4bf1-bb4f-a156191a8ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:00.454185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:00.456427 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f13c7-b348-4249-89e8-6199dbe7b49c None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:00.460001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:00.503374 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1070/2126] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:00.511975 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48753993-ac41-4409-8750-aac46ed21837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:00.512547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48753993-ac41-4409-8750-aac46ed21837 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:00.512843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48753993-ac41-4409-8750-aac46ed21837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:00.513099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48753993-ac41-4409-8750-aac46ed21837 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:00.513720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48753993-ac41-4409-8750-aac46ed21837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:00.513859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831233e-1a26-4bf1-bb4f-a156191a8ff1 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:00.515184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4831233e-1a26-4bf1-bb4f-a156191a8ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:00.515608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48753993-ac41-4409-8750-aac46ed21837 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:00.572500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831233e-1a26-4bf1-bb4f-a156191a8ff1 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:00.576771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca862400-c013-4677-9a84-5a225cfb8d25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:00.580306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca862400-c013-4677-9a84-5a225cfb8d25 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:00.580306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca862400-c013-4677-9a84-5a225cfb8d25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:00.580306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca862400-c013-4677-9a84-5a225cfb8d25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:00.586690 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca862400-c013-4677-9a84-5a225cfb8d25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:00.588107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca862400-c013-4677-9a84-5a225cfb8d25 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:00.597492 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1071/2127] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:50:00.609365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1057/2128] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:50:00 2022] POST /identity/v3/projects => generated 377 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:50:00.616209 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca862400-c013-4677-9a84-5a225cfb8d25 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:00.624708 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca862400-c013-4677-9a84-5a225cfb8d25 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:00.680750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad35e2-a42b-40c8-998c-596ee739fa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:00.681447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad35e2-a42b-40c8-998c-596ee739fa85 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:00.681727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad35e2-a42b-40c8-998c-596ee739fa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:00.681994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad35e2-a42b-40c8-998c-596ee739fa85 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:00.682641 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad35e2-a42b-40c8-998c-596ee739fa85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:00.685134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad35e2-a42b-40c8-998c-596ee739fa85 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:00.789273 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1072/2129] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:50:00 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:50:00.790777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48753993-ac41-4409-8750-aac46ed21837 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:00.801040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48753993-ac41-4409-8750-aac46ed21837 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:00.805659 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a579e6-9fe6-4f5e-8e52-06630952b040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:00.806260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a579e6-9fe6-4f5e-8e52-06630952b040 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:00.806537 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a579e6-9fe6-4f5e-8e52-06630952b040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:00.806806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a579e6-9fe6-4f5e-8e52-06630952b040 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:00.807439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a579e6-9fe6-4f5e-8e52-06630952b040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:00.809630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a579e6-9fe6-4f5e-8e52-06630952b040 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:00.824546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1058/2130] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:00.825531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad35e2-a42b-40c8-998c-596ee739fa85 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:00.836263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad35e2-a42b-40c8-998c-596ee739fa85 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:00.840981 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004e16b-0a46-483d-a06f-dd3d728e7e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:00.841579 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004e16b-0a46-483d-a06f-dd3d728e7e0b admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:00.841857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004e16b-0a46-483d-a06f-dd3d728e7e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:00.842126 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004e16b-0a46-483d-a06f-dd3d728e7e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:00.842795 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004e16b-0a46-483d-a06f-dd3d728e7e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:00.844384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004e16b-0a46-483d-a06f-dd3d728e7e0b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:00.854364 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1073/2131] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:00.859227 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a579e6-9fe6-4f5e-8e52-06630952b040 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:00.869944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a579e6-9fe6-4f5e-8e52-06630952b040 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:00.875392 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b70da9-7b79-49bf-8b84-f2ecb7806336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:00.876140 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b70da9-7b79-49bf-8b84-f2ecb7806336 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:00.876525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b70da9-7b79-49bf-8b84-f2ecb7806336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:00.876872 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b70da9-7b79-49bf-8b84-f2ecb7806336 admin admin] PATH_INFO: `/v3/projects/eeaf0dc6c1f84183976a4ab5b83d077e/users/11292a99199f4eaaa46057b6a350520c/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:00.894173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b70da9-7b79-49bf-8b84-f2ecb7806336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeaf0dc6c1f84183976a4ab5b83d077e, user_id=11292a99199f4eaaa46057b6a350520c, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:00.896972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b70da9-7b79-49bf-8b84-f2ecb7806336 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:01.000188 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1059/2132] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:00 2022] PUT /identity/v3/projects/eeaf0dc6c1f84183976a4ab5b83d077e/users/11292a99199f4eaaa46057b6a350520c/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:01.010484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f04c8e-0508-4a77-a294-8248ec2bc2d9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:01.010773 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f04c8e-0508-4a77-a294-8248ec2bc2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:01.011049 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f04c8e-0508-4a77-a294-8248ec2bc2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:01.037835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f7f04c8e-0508-4a77-a294-8248ec2bc2d9 None None] MFA Rules not processed for user `11292a99199f4eaaa46057b6a350520c`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:01.039020 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f7f04c8e-0508-4a77-a294-8248ec2bc2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:01.059888 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b70da9-7b79-49bf-8b84-f2ecb7806336 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:01.062032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-67b70da9-7b79-49bf-8b84-f2ecb7806336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:01.092916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:01.153416 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1074/2133] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:01.157958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f04c8e-0508-4a77-a294-8248ec2bc2d9 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:01.169169 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b70da9-7b79-49bf-8b84-f2ecb7806336 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:01.169800 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f04c8e-0508-4a77-a294-8248ec2bc2d9 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:01.174494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b8e6f-aa6a-4c2b-b6ea-f38d86857fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:01.175099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b8e6f-aa6a-4c2b-b6ea-f38d86857fdf admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:01.175217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4767f54-aaa3-464d-a7d0-95c79205d688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:01.175378 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b8e6f-aa6a-4c2b-b6ea-f38d86857fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:01.175653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b8e6f-aa6a-4c2b-b6ea-f38d86857fdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:01.176127 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4767f54-aaa3-464d-a7d0-95c79205d688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:01.176279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b8e6f-aa6a-4c2b-b6ea-f38d86857fdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:01.176619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4767f54-aaa3-464d-a7d0-95c79205d688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:01.177032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4767f54-aaa3-464d-a7d0-95c79205d688 admin admin] PATH_INFO: `/v3/projects/cb469a5b5781412887cd32a7dc805260` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:01.178199 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b8e6f-aa6a-4c2b-b6ea-f38d86857fdf admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:01.178574 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4767f54-aaa3-464d-a7d0-95c79205d688 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb469a5b5781412887cd32a7dc805260)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:01.180826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4767f54-aaa3-464d-a7d0-95c79205d688 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:01.301826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1075/2134] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:50:01 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:50:01.303116 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b8e6f-aa6a-4c2b-b6ea-f38d86857fdf admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:01.332238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b4767f54-aaa3-464d-a7d0-95c79205d688 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb469a5b5781412887cd32a7dc805260', 'request_id': 'req-b4767f54-aaa3-464d-a7d0-95c79205d688'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:01.341290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b4767f54-aaa3-464d-a7d0-95c79205d688 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb469a5b5781412887cd32a7dc805260', 'request_id': 'req-b4767f54-aaa3-464d-a7d0-95c79205d688'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:01.346216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:01.349634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b4767f54-aaa3-464d-a7d0-95c79205d688 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb469a5b5781412887cd32a7dc805260', 'request_id': 'req-b4767f54-aaa3-464d-a7d0-95c79205d688'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:01.395799 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b8e6f-aa6a-4c2b-b6ea-f38d86857fdf admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:01.440486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b4767f54-aaa3-464d-a7d0-95c79205d688 admin admin] The token cache is being invalidate because project cb469a5b5781412887cd32a7dc805260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:01.440778 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b4767f54-aaa3-464d-a7d0-95c79205d688 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:01.441426 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-b4767f54-aaa3-464d-a7d0-95c79205d688 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:01.442933 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1060/2135] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:01 2022] DELETE /identity/v3/projects/cb469a5b5781412887cd32a7dc805260 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:50:01.447283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4767f54-aaa3-464d-a7d0-95c79205d688 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:01.449211 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b4767f54-aaa3-464d-a7d0-95c79205d688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:01.465207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9ef81-6f6b-451b-b979-aabc8f5ea740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:01.465908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9ef81-6f6b-451b-b979-aabc8f5ea740 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:01.466137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9ef81-6f6b-451b-b979-aabc8f5ea740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:01.466411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9ef81-6f6b-451b-b979-aabc8f5ea740 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:01.467037 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9ef81-6f6b-451b-b979-aabc8f5ea740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:01.469017 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9ef81-6f6b-451b-b979-aabc8f5ea740 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:01.512902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4767f54-aaa3-464d-a7d0-95c79205d688 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:01.517554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2ead8-c707-404d-9dfa-7c4a9bb5eb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:01.518202 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2ead8-c707-404d-9dfa-7c4a9bb5eb9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:01.518482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2ead8-c707-404d-9dfa-7c4a9bb5eb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:01.518754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2ead8-c707-404d-9dfa-7c4a9bb5eb9d admin admin] PATH_INFO: `/v3/users/919a84a62d5541e891b0deabef96cde2` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:01.519875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2ead8-c707-404d-9dfa-7c4a9bb5eb9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=919a84a62d5541e891b0deabef96cde2)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:01.521806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2ead8-c707-404d-9dfa-7c4a9bb5eb9d admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:01.571024 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1076/2136] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:50:01 2022] POST /identity/v3/users => generated 295 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:01.571921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9ef81-6f6b-451b-b979-aabc8f5ea740 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:01.573529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-11a9ef81-6f6b-451b-b979-aabc8f5ea740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:01.585132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9ef81-6f6b-451b-b979-aabc8f5ea740 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:01.590547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f9983-c1c8-4e7f-bc36-2d8a12b17ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:01.591134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f9983-c1c8-4e7f-bc36-2d8a12b17ebf admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:01.591415 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f9983-c1c8-4e7f-bc36-2d8a12b17ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:01.591689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f9983-c1c8-4e7f-bc36-2d8a12b17ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:01.592355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f9983-c1c8-4e7f-bc36-2d8a12b17ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:01.593964 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f9983-c1c8-4e7f-bc36-2d8a12b17ebf admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:01.607300 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1077/2137] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:50:01.608667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f9983-c1c8-4e7f-bc36-2d8a12b17ebf admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:01.618133 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f9983-c1c8-4e7f-bc36-2d8a12b17ebf admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:01.623149 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42d559-f00f-4779-9a2d-e48425ecf5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:01.623813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42d559-f00f-4779-9a2d-e48425ecf5c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:01.624094 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42d559-f00f-4779-9a2d-e48425ecf5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:01.624370 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42d559-f00f-4779-9a2d-e48425ecf5c2 admin admin] PATH_INFO: `/v3/projects/98bfff925dc540afa2e6feef3acdb79d/users/2b782eb5b6f94ae0a3284d745fe0c8ee/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:01.639435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42d559-f00f-4779-9a2d-e48425ecf5c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98bfff925dc540afa2e6feef3acdb79d, user_id=2b782eb5b6f94ae0a3284d745fe0c8ee, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:01.642098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42d559-f00f-4779-9a2d-e48425ecf5c2 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:01.711256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-33d2ead8-c707-404d-9dfa-7c4a9bb5eb9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '919a84a62d5541e891b0deabef96cde2'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:01.711996 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-33d2ead8-c707-404d-9dfa-7c4a9bb5eb9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '919a84a62d5541e891b0deabef96cde2'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:01.712710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-33d2ead8-c707-404d-9dfa-7c4a9bb5eb9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '919a84a62d5541e891b0deabef96cde2'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:01.725199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-33d2ead8-c707-404d-9dfa-7c4a9bb5eb9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '919a84a62d5541e891b0deabef96cde2'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:01.725904 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1078/2138] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:01 2022] PUT /identity/v3/projects/98bfff925dc540afa2e6feef3acdb79d/users/2b782eb5b6f94ae0a3284d745fe0c8ee/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:50:01.729832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42d559-f00f-4779-9a2d-e48425ecf5c2 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:01.731427 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dc42d559-f00f-4779-9a2d-e48425ecf5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:01.741978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:01.800239 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42d559-f00f-4779-9a2d-e48425ecf5c2 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:01.804810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bd397-5a4c-49c1-8878-2a0d7c794d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:01.805452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bd397-5a4c-49c1-8878-2a0d7c794d91 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:01.805731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bd397-5a4c-49c1-8878-2a0d7c794d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:01.806001 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bd397-5a4c-49c1-8878-2a0d7c794d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:01.806781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bd397-5a4c-49c1-8878-2a0d7c794d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:01.808369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bd397-5a4c-49c1-8878-2a0d7c794d91 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:01.817826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1079/2139] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:50:01.821760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bd397-5a4c-49c1-8878-2a0d7c794d91 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:01.825434 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-33d2ead8-c707-404d-9dfa-7c4a9bb5eb9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '919a84a62d5541e891b0deabef96cde2'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:01.831832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bd397-5a4c-49c1-8878-2a0d7c794d91 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:01.836748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0c08e-257d-4da6-a2d6-d2bf67678c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:01.840060 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0c08e-257d-4da6-a2d6-d2bf67678c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:01.840060 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0c08e-257d-4da6-a2d6-d2bf67678c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:01.840060 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0c08e-257d-4da6-a2d6-d2bf67678c8b admin admin] PATH_INFO: `/v3/projects/98bfff925dc540afa2e6feef3acdb79d/users/2b782eb5b6f94ae0a3284d745fe0c8ee/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:01.840901 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e0c08e-257d-4da6-a2d6-d2bf67678c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=98bfff925dc540afa2e6feef3acdb79d, user_id=2b782eb5b6f94ae0a3284d745fe0c8ee, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:01.843681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e0c08e-257d-4da6-a2d6-d2bf67678c8b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:01.845321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1061/2140] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:01 2022] DELETE /identity/v3/users/919a84a62d5541e891b0deabef96cde2 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:01.854708 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1080/2141] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:01 2022] PUT /identity/v3/projects/98bfff925dc540afa2e6feef3acdb79d/users/2b782eb5b6f94ae0a3284d745fe0c8ee/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:50:01.924618 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb9c71-b2cc-4f4d-b172-ec2b2dcd4f58 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:01.924911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb9c71-b2cc-4f4d-b172-ec2b2dcd4f58 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:01.925193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb9c71-b2cc-4f4d-b172-ec2b2dcd4f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:01.953955 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-26fb9c71-b2cc-4f4d-b172-ec2b2dcd4f58 None None] MFA Rules not processed for user `2b782eb5b6f94ae0a3284d745fe0c8ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:01.955172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-26fb9c71-b2cc-4f4d-b172-ec2b2dcd4f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:02.005847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:02.023444 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1062/2142] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:02.035110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0c08e-257d-4da6-a2d6-d2bf67678c8b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:02.036813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-71e0c08e-257d-4da6-a2d6-d2bf67678c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:02.100446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0c08e-257d-4da6-a2d6-d2bf67678c8b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:02.104101 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255581a-8764-4fe1-80a9-35f46c5c19c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:02.104954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255581a-8764-4fe1-80a9-35f46c5c19c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:02.105240 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255581a-8764-4fe1-80a9-35f46c5c19c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:02.105518 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255581a-8764-4fe1-80a9-35f46c5c19c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:02.107314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6255581a-8764-4fe1-80a9-35f46c5c19c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:02.176325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255581a-8764-4fe1-80a9-35f46c5c19c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:02.177787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255581a-8764-4fe1-80a9-35f46c5c19c4 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:02.189233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1081/2143] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:02.358450 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb9c71-b2cc-4f4d-b172-ec2b2dcd4f58 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:02.359845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-26fb9c71-b2cc-4f4d-b172-ec2b2dcd4f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:02.439987 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb9c71-b2cc-4f4d-b172-ec2b2dcd4f58 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:02.444852 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e4585-f1fc-445e-a45e-cbe476a3c675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:02.445552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e4585-f1fc-445e-a45e-cbe476a3c675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:02.445775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e4585-f1fc-445e-a45e-cbe476a3c675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:02.446053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e4585-f1fc-445e-a45e-cbe476a3c675 admin admin] PATH_INFO: `/v3/projects/07105c938ec14a66bd933f8c1efada01` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:02.447250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e4585-f1fc-445e-a45e-cbe476a3c675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07105c938ec14a66bd933f8c1efada01)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:02.449231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e4585-f1fc-445e-a45e-cbe476a3c675 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:02.596342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ff9e4585-f1fc-445e-a45e-cbe476a3c675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07105c938ec14a66bd933f8c1efada01', 'request_id': 'req-ff9e4585-f1fc-445e-a45e-cbe476a3c675'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:02.623035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ff9e4585-f1fc-445e-a45e-cbe476a3c675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07105c938ec14a66bd933f8c1efada01', 'request_id': 'req-ff9e4585-f1fc-445e-a45e-cbe476a3c675'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:02.628870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ff9e4585-f1fc-445e-a45e-cbe476a3c675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07105c938ec14a66bd933f8c1efada01', 'request_id': 'req-ff9e4585-f1fc-445e-a45e-cbe476a3c675'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:02.701048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ff9e4585-f1fc-445e-a45e-cbe476a3c675 admin admin] The token cache is being invalidate because project 07105c938ec14a66bd933f8c1efada01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:02.701339 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ff9e4585-f1fc-445e-a45e-cbe476a3c675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:02.701957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ff9e4585-f1fc-445e-a45e-cbe476a3c675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:02.703455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1063/2144] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:02 2022] DELETE /identity/v3/projects/07105c938ec14a66bd933f8c1efada01 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:02.708389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a10f25-31d1-4be9-8568-29142454e2e0 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:02.708897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a10f25-31d1-4be9-8568-29142454e2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:02.709029 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a10f25-31d1-4be9-8568-29142454e2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:02.729675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-05a10f25-31d1-4be9-8568-29142454e2e0 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:02.731493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-05a10f25-31d1-4be9-8568-29142454e2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:02.773062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:02.790992 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1082/2145] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:02.806065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d289a-c896-493a-992e-42eae72704a6 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:02.806590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d289a-c896-493a-992e-42eae72704a6 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:02.806878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d289a-c896-493a-992e-42eae72704a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:02.826935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-e07d289a-c896-493a-992e-42eae72704a6 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:02.828118 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e07d289a-c896-493a-992e-42eae72704a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:02.836472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1064/2146] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:02.841687 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a10f25-31d1-4be9-8568-29142454e2e0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:02.851694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a10f25-31d1-4be9-8568-29142454e2e0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:02.856914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ad443-db68-4b53-984a-81e437f633f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:02.857556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ad443-db68-4b53-984a-81e437f633f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:02.857861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ad443-db68-4b53-984a-81e437f633f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:02.858103 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ad443-db68-4b53-984a-81e437f633f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:02.858856 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ad443-db68-4b53-984a-81e437f633f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:50:02.859191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ad443-db68-4b53-984a-81e437f633f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:02.860828 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ad443-db68-4b53-984a-81e437f633f0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:02.876980 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1083/2147] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:50:02.878382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d289a-c896-493a-992e-42eae72704a6 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:02.887707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d289a-c896-493a-992e-42eae72704a6 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:02.892661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90910b2c-cdf7-42db-9ac1-3b0e0eab905a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:02.893275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90910b2c-cdf7-42db-9ac1-3b0e0eab905a admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:02.893557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90910b2c-cdf7-42db-9ac1-3b0e0eab905a admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:02.893827 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90910b2c-cdf7-42db-9ac1-3b0e0eab905a admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:02.894504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90910b2c-cdf7-42db-9ac1-3b0e0eab905a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:02.896425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90910b2c-cdf7-42db-9ac1-3b0e0eab905a admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:02.986447 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1065/2148] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:50:02 2022] POST /identity/v3/projects => generated 401 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:02.990489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ad443-db68-4b53-984a-81e437f633f0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:03.000514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ad443-db68-4b53-984a-81e437f633f0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:03.057581 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04943e7-4af1-4c48-ae5e-d0385729e03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:03.058186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04943e7-4af1-4c48-ae5e-d0385729e03f admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:03.058464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04943e7-4af1-4c48-ae5e-d0385729e03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:03.058735 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04943e7-4af1-4c48-ae5e-d0385729e03f admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:03.059367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04943e7-4af1-4c48-ae5e-d0385729e03f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:03.061293 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04943e7-4af1-4c48-ae5e-d0385729e03f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:03.162036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1084/2149] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:50:02 2022] POST /identity/v3/users => generated 301 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:50:03.166072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90910b2c-cdf7-42db-9ac1-3b0e0eab905a admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:03.176451 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90910b2c-cdf7-42db-9ac1-3b0e0eab905a admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:03.181133 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ded88b-98b7-4f08-90a9-81c3b8e1bce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:03.183138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ded88b-98b7-4f08-90a9-81c3b8e1bce9 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:03.183138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ded88b-98b7-4f08-90a9-81c3b8e1bce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:03.183138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ded88b-98b7-4f08-90a9-81c3b8e1bce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:03.183138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ded88b-98b7-4f08-90a9-81c3b8e1bce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:03.184588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ded88b-98b7-4f08-90a9-81c3b8e1bce9 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:03.194829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1066/2150] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:03.199167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04943e7-4af1-4c48-ae5e-d0385729e03f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:03.209155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04943e7-4af1-4c48-ae5e-d0385729e03f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:03.213742 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15206624-5caf-41fe-9039-3adc83ad5e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:03.215008 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15206624-5caf-41fe-9039-3adc83ad5e09 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:03.215293 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15206624-5caf-41fe-9039-3adc83ad5e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:03.215563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15206624-5caf-41fe-9039-3adc83ad5e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:03.216237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15206624-5caf-41fe-9039-3adc83ad5e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:03.217885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15206624-5caf-41fe-9039-3adc83ad5e09 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:03.229012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1085/2151] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:03.233743 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ded88b-98b7-4f08-90a9-81c3b8e1bce9 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:03.243866 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ded88b-98b7-4f08-90a9-81c3b8e1bce9 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:03.249150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06d621-9afe-4945-bdb7-d2671691404d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:03.249866 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06d621-9afe-4945-bdb7-d2671691404d admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:03.250153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06d621-9afe-4945-bdb7-d2671691404d admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:03.250428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06d621-9afe-4945-bdb7-d2671691404d admin admin] PATH_INFO: `/v3/projects/d4e437e60152433a96a35f23d85810e5/users/d66efa774a8741f1909c3cb45a8b4e58/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:03.266181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06d621-9afe-4945-bdb7-d2671691404d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e437e60152433a96a35f23d85810e5, user_id=d66efa774a8741f1909c3cb45a8b4e58, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:03.269050 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06d621-9afe-4945-bdb7-d2671691404d admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:03.365467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1067/2152] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:03 2022] PUT /identity/v3/projects/d4e437e60152433a96a35f23d85810e5/users/d66efa774a8741f1909c3cb45a8b4e58/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:03.377531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea8d8e-ad81-4344-98f1-233f371f17cf None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:03.378113 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea8d8e-ad81-4344-98f1-233f371f17cf None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:03.378397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea8d8e-ad81-4344-98f1-233f371f17cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:03.407128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-81ea8d8e-ad81-4344-98f1-233f371f17cf None None] MFA Rules not processed for user `d66efa774a8741f1909c3cb45a8b4e58`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:03.408410 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-81ea8d8e-ad81-4344-98f1-233f371f17cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:03.471759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1086/2153] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:03 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:03.476228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06d621-9afe-4945-bdb7-d2671691404d admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:03.496442 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:03.543031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06d621-9afe-4945-bdb7-d2671691404d admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:03.548521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9241c3-bbd7-4fec-8da4-3c8fe0625d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:03.549124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9241c3-bbd7-4fec-8da4-3c8fe0625d77 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:03.549400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9241c3-bbd7-4fec-8da4-3c8fe0625d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:03.549666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9241c3-bbd7-4fec-8da4-3c8fe0625d77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:03.550293 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9241c3-bbd7-4fec-8da4-3c8fe0625d77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:03.552235 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9241c3-bbd7-4fec-8da4-3c8fe0625d77 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:03.649993 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1068/2154] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:50:03 2022] POST /identity/v3/projects => generated 399 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:03.651373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea8d8e-ad81-4344-98f1-233f371f17cf None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:03.680072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea8d8e-ad81-4344-98f1-233f371f17cf None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:03.724492 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:03.747430 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621bbaa9-5416-440b-8803-5a39d52e0337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:03.748395 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bbaa9-5416-440b-8803-5a39d52e0337 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:03.748948 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bbaa9-5416-440b-8803-5a39d52e0337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:03.749406 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bbaa9-5416-440b-8803-5a39d52e0337 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:03.750355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621bbaa9-5416-440b-8803-5a39d52e0337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:03.752669 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621bbaa9-5416-440b-8803-5a39d52e0337 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:03.897806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1087/2155] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:50:03 2022] POST /identity/v3/users => generated 306 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:03.912747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9241c3-bbd7-4fec-8da4-3c8fe0625d77 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:03.930922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9241c3-bbd7-4fec-8da4-3c8fe0625d77 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:03.937519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b051b44-5e55-4be0-9723-cd5a89c81c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:03.938143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b051b44-5e55-4be0-9723-cd5a89c81c82 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:03.938426 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b051b44-5e55-4be0-9723-cd5a89c81c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:03.938699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b051b44-5e55-4be0-9723-cd5a89c81c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:03.939383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b051b44-5e55-4be0-9723-cd5a89c81c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:03.941056 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b051b44-5e55-4be0-9723-cd5a89c81c82 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:03.953110 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1069/2156] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:03.957238 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621bbaa9-5416-440b-8803-5a39d52e0337 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:03.967476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621bbaa9-5416-440b-8803-5a39d52e0337 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:03.972529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5754d0-4c7d-4d7c-9d61-67561b5c9381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:03.973190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5754d0-4c7d-4d7c-9d61-67561b5c9381 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:03.973472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5754d0-4c7d-4d7c-9d61-67561b5c9381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:03.973747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5754d0-4c7d-4d7c-9d61-67561b5c9381 admin admin] PATH_INFO: `/v3/projects/0a392434b5b147aa8c62a7bf8bf1586b/users/f961fd1772f14524bbc25a35f4cc0e81/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:03.989668 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5754d0-4c7d-4d7c-9d61-67561b5c9381 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a392434b5b147aa8c62a7bf8bf1586b, user_id=f961fd1772f14524bbc25a35f4cc0e81, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:03.992261 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5754d0-4c7d-4d7c-9d61-67561b5c9381 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:04.083054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1088/2157] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:03 2022] PUT /identity/v3/projects/0a392434b5b147aa8c62a7bf8bf1586b/users/f961fd1772f14524bbc25a35f4cc0e81/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:04.086966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b051b44-5e55-4be0-9723-cd5a89c81c82 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:04.098237 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b051b44-5e55-4be0-9723-cd5a89c81c82 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:04.164388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65007f3-fc5b-4bde-9367-98e63ea420f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:04.165309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65007f3-fc5b-4bde-9367-98e63ea420f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:04.165935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65007f3-fc5b-4bde-9367-98e63ea420f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:04.166213 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65007f3-fc5b-4bde-9367-98e63ea420f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:04.166860 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65007f3-fc5b-4bde-9367-98e63ea420f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:04.168491 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65007f3-fc5b-4bde-9367-98e63ea420f2 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:04.179023 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1070/2158] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:04.183158 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5754d0-4c7d-4d7c-9d61-67561b5c9381 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:04.193418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5754d0-4c7d-4d7c-9d61-67561b5c9381 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:04.198279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f1412-13a0-4b53-b527-44653b7272e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:04.208887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f1412-13a0-4b53-b527-44653b7272e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:04.208887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f1412-13a0-4b53-b527-44653b7272e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:04.208887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f1412-13a0-4b53-b527-44653b7272e9 admin admin] PATH_INFO: `/v3/projects/0a392434b5b147aa8c62a7bf8bf1586b/users/f961fd1772f14524bbc25a35f4cc0e81/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:04.208887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f1412-13a0-4b53-b527-44653b7272e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a392434b5b147aa8c62a7bf8bf1586b, user_id=f961fd1772f14524bbc25a35f4cc0e81, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:04.208887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f1412-13a0-4b53-b527-44653b7272e9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:04.217505 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1089/2159] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:04 2022] PUT /identity/v3/projects/0a392434b5b147aa8c62a7bf8bf1586b/users/f961fd1772f14524bbc25a35f4cc0e81/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:50:04.227757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6119a-4fe0-4efc-a816-3d4a6861853a None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:04.228049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6119a-4fe0-4efc-a816-3d4a6861853a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:04.228325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6119a-4fe0-4efc-a816-3d4a6861853a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:04.258557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-5cc6119a-4fe0-4efc-a816-3d4a6861853a None None] MFA Rules not processed for user `f961fd1772f14524bbc25a35f4cc0e81`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:04.259760 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-5cc6119a-4fe0-4efc-a816-3d4a6861853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:04.332793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1071/2160] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:04 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:04.344750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f1412-13a0-4b53-b527-44653b7272e9 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:04.345478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d05f1412-13a0-4b53-b527-44653b7272e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:04.406875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f1412-13a0-4b53-b527-44653b7272e9 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:04.410806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb751703-77ff-4695-a9f1-4593269adecc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:04.411422 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb751703-77ff-4695-a9f1-4593269adecc service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:04.411704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb751703-77ff-4695-a9f1-4593269adecc service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:04.412189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb751703-77ff-4695-a9f1-4593269adecc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:04.421902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb751703-77ff-4695-a9f1-4593269adecc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:04.423435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb751703-77ff-4695-a9f1-4593269adecc service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:04.435306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1090/2161] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:04 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:04.631407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6119a-4fe0-4efc-a816-3d4a6861853a None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:04.640981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6119a-4fe0-4efc-a816-3d4a6861853a None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:04.644551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9bb09-b824-43fe-b4f3-08782060055f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:04.645148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9bb09-b824-43fe-b4f3-08782060055f service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:04.645425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9bb09-b824-43fe-b4f3-08782060055f service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:04.645696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9bb09-b824-43fe-b4f3-08782060055f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:04.655046 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:04.702584 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9bb09-b824-43fe-b4f3-08782060055f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:04.705638 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9bb09-b824-43fe-b4f3-08782060055f service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:04.712768 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1072/2162] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:04 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:05.184108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb751703-77ff-4695-a9f1-4593269adecc service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:05.185614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bb751703-77ff-4695-a9f1-4593269adecc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:05.196958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb751703-77ff-4695-a9f1-4593269adecc service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:05.200638 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1dbff-c3a8-4424-b987-f436f6410fe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:05.201239 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1dbff-c3a8-4424-b987-f436f6410fe7 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:05.201524 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1dbff-c3a8-4424-b987-f436f6410fe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:05.201799 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1dbff-c3a8-4424-b987-f436f6410fe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:05.203486 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c3f1dbff-c3a8-4424-b987-f436f6410fe7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:05.262186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1dbff-c3a8-4424-b987-f436f6410fe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:05.263621 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1dbff-c3a8-4424-b987-f436f6410fe7 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:05.274068 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1091/2163] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:09.914463 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9bb09-b824-43fe-b4f3-08782060055f service cinder] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:09.924359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9bb09-b824-43fe-b4f3-08782060055f service cinder] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:09.928054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db100df-600f-466e-960a-ae38a8c4e4ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:09.928699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db100df-600f-466e-960a-ae38a8c4e4ab service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:09.928976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db100df-600f-466e-960a-ae38a8c4e4ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:09.929254 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db100df-600f-466e-960a-ae38a8c4e4ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:09.931027 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6db100df-600f-466e-960a-ae38a8c4e4ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:09.997657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db100df-600f-466e-960a-ae38a8c4e4ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:09.999667 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db100df-600f-466e-960a-ae38a8c4e4ab service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:10.009503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1073/2164] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:15.269447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:17.450920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1dbff-c3a8-4424-b987-f436f6410fe7 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:17.515174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1dbff-c3a8-4424-b987-f436f6410fe7 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:17.519629 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a18922-b7c6-4cd0-8fb2-813d512afbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:17.520250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18922-b7c6-4cd0-8fb2-813d512afbaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:17.520563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18922-b7c6-4cd0-8fb2-813d512afbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:17.520841 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18922-b7c6-4cd0-8fb2-813d512afbaf admin admin] PATH_INFO: `/v3/users/d66efa774a8741f1909c3cb45a8b4e58` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:17.521941 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a18922-b7c6-4cd0-8fb2-813d512afbaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d66efa774a8741f1909c3cb45a8b4e58)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:17.524544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a18922-b7c6-4cd0-8fb2-813d512afbaf admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:17.710394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-94a18922-b7c6-4cd0-8fb2-813d512afbaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd66efa774a8741f1909c3cb45a8b4e58'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:17.711085 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-94a18922-b7c6-4cd0-8fb2-813d512afbaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd66efa774a8741f1909c3cb45a8b4e58'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:17.711694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-94a18922-b7c6-4cd0-8fb2-813d512afbaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd66efa774a8741f1909c3cb45a8b4e58'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:17.784845 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-94a18922-b7c6-4cd0-8fb2-813d512afbaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd66efa774a8741f1909c3cb45a8b4e58'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:17.792111 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-94a18922-b7c6-4cd0-8fb2-813d512afbaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd66efa774a8741f1909c3cb45a8b4e58'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:17.811606 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1092/2165] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:17 2022] DELETE /identity/v3/users/d66efa774a8741f1909c3cb45a8b4e58 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:17.816166 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad6a6a-864c-4ff4-a2c4-b72f261c2740 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:17.816684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad6a6a-864c-4ff4-a2c4-b72f261c2740 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:17.816787 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad6a6a-864c-4ff4-a2c4-b72f261c2740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:17.839451 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-34ad6a6a-864c-4ff4-a2c4-b72f261c2740 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:17.839451 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-34ad6a6a-864c-4ff4-a2c4-b72f261c2740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:17.895699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1074/2166] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:17.905818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a18922-b7c6-4cd0-8fb2-813d512afbaf admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:17.907234 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-94a18922-b7c6-4cd0-8fb2-813d512afbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:17.962488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a18922-b7c6-4cd0-8fb2-813d512afbaf admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:17.966091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfcfa91-c702-42ee-a9b6-76d0041cc70e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:17.966690 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcfa91-c702-42ee-a9b6-76d0041cc70e service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:17.967471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcfa91-c702-42ee-a9b6-76d0041cc70e service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:17.967741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcfa91-c702-42ee-a9b6-76d0041cc70e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:17.976394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfcfa91-c702-42ee-a9b6-76d0041cc70e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:17.977827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfcfa91-c702-42ee-a9b6-76d0041cc70e service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:17.987502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1093/2167] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:18.423931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad6a6a-864c-4ff4-a2c4-b72f261c2740 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:18.423931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-34ad6a6a-864c-4ff4-a2c4-b72f261c2740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:18.433154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad6a6a-864c-4ff4-a2c4-b72f261c2740 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:18.439447 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886baf61-a280-44d2-a59f-9141417dcc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:18.440164 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886baf61-a280-44d2-a59f-9141417dcc50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:18.440570 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886baf61-a280-44d2-a59f-9141417dcc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:18.440939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886baf61-a280-44d2-a59f-9141417dcc50 admin admin] PATH_INFO: `/v3/projects/d4e437e60152433a96a35f23d85810e5` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:18.442257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886baf61-a280-44d2-a59f-9141417dcc50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4e437e60152433a96a35f23d85810e5)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:18.444782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886baf61-a280-44d2-a59f-9141417dcc50 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:18.565519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:18.571854 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-886baf61-a280-44d2-a59f-9141417dcc50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4e437e60152433a96a35f23d85810e5', 'request_id': 'req-886baf61-a280-44d2-a59f-9141417dcc50'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:18.579131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-886baf61-a280-44d2-a59f-9141417dcc50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4e437e60152433a96a35f23d85810e5', 'request_id': 'req-886baf61-a280-44d2-a59f-9141417dcc50'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:18.585039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-886baf61-a280-44d2-a59f-9141417dcc50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4e437e60152433a96a35f23d85810e5', 'request_id': 'req-886baf61-a280-44d2-a59f-9141417dcc50'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:18.649472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-886baf61-a280-44d2-a59f-9141417dcc50 admin admin] The token cache is being invalidate because project d4e437e60152433a96a35f23d85810e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:18.649760 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-886baf61-a280-44d2-a59f-9141417dcc50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:18.650417 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-886baf61-a280-44d2-a59f-9141417dcc50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:18.652136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1075/2168] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:18 2022] DELETE /identity/v3/projects/d4e437e60152433a96a35f23d85810e5 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:18.655866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfcfa91-c702-42ee-a9b6-76d0041cc70e service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:18.665290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfcfa91-c702-42ee-a9b6-76d0041cc70e service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:18.682499 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:18.720109 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04932a96-7143-48b5-9512-7ece0abac0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:18.720772 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04932a96-7143-48b5-9512-7ece0abac0ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:18.721134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04932a96-7143-48b5-9512-7ece0abac0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:18.721409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04932a96-7143-48b5-9512-7ece0abac0ee admin admin] PATH_INFO: `/v3/users/f961fd1772f14524bbc25a35f4cc0e81` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:18.722528 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04932a96-7143-48b5-9512-7ece0abac0ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=f961fd1772f14524bbc25a35f4cc0e81)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:18.724546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04932a96-7143-48b5-9512-7ece0abac0ee admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:18.952948 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-04932a96-7143-48b5-9512-7ece0abac0ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f961fd1772f14524bbc25a35f4cc0e81'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:18.953600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-04932a96-7143-48b5-9512-7ece0abac0ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f961fd1772f14524bbc25a35f4cc0e81'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:18.954197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-04932a96-7143-48b5-9512-7ece0abac0ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f961fd1772f14524bbc25a35f4cc0e81'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:19.062785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-04932a96-7143-48b5-9512-7ece0abac0ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f961fd1772f14524bbc25a35f4cc0e81'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:19.070572 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-04932a96-7143-48b5-9512-7ece0abac0ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f961fd1772f14524bbc25a35f4cc0e81'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:19.087933 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1094/2169] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:18 2022] DELETE /identity/v3/users/f961fd1772f14524bbc25a35f4cc0e81 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:19.574760 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886baf61-a280-44d2-a59f-9141417dcc50 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:19.577466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-886baf61-a280-44d2-a59f-9141417dcc50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:19.603238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886baf61-a280-44d2-a59f-9141417dcc50 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:19.718859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcfb0a-ca39-4a51-8165-b7564041b157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:19.719731 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcfb0a-ca39-4a51-8165-b7564041b157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:19.720052 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcfb0a-ca39-4a51-8165-b7564041b157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:19.720564 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcfb0a-ca39-4a51-8165-b7564041b157 admin admin] PATH_INFO: `/v3/projects/0a392434b5b147aa8c62a7bf8bf1586b` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:19.722131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcfb0a-ca39-4a51-8165-b7564041b157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a392434b5b147aa8c62a7bf8bf1586b)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:19.724425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcfb0a-ca39-4a51-8165-b7564041b157 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:19.883145 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4dfcfb0a-ca39-4a51-8165-b7564041b157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a392434b5b147aa8c62a7bf8bf1586b', 'request_id': 'req-4dfcfb0a-ca39-4a51-8165-b7564041b157'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:19.889881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4dfcfb0a-ca39-4a51-8165-b7564041b157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a392434b5b147aa8c62a7bf8bf1586b', 'request_id': 'req-4dfcfb0a-ca39-4a51-8165-b7564041b157'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:19.896209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4dfcfb0a-ca39-4a51-8165-b7564041b157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a392434b5b147aa8c62a7bf8bf1586b', 'request_id': 'req-4dfcfb0a-ca39-4a51-8165-b7564041b157'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:20.027039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4dfcfb0a-ca39-4a51-8165-b7564041b157 admin admin] The token cache is being invalidate because project 0a392434b5b147aa8c62a7bf8bf1586b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:20.027361 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4dfcfb0a-ca39-4a51-8165-b7564041b157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:20.028033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4dfcfb0a-ca39-4a51-8165-b7564041b157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:20.030424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1076/2170] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:19 2022] DELETE /identity/v3/projects/0a392434b5b147aa8c62a7bf8bf1586b => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:20.035483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd1791-fe60-4834-8ba0-4d069926f9cd None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:20.035944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd1791-fe60-4834-8ba0-4d069926f9cd None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:20.036040 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd1791-fe60-4834-8ba0-4d069926f9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:20.056483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e1dd1791-fe60-4834-8ba0-4d069926f9cd None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:20.057790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e1dd1791-fe60-4834-8ba0-4d069926f9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:20.178597 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1095/2171] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:20.186740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8591f-dd1c-4728-9da0-9b5e41433f09 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:20.187030 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8591f-dd1c-4728-9da0-9b5e41433f09 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:20.187311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8591f-dd1c-4728-9da0-9b5e41433f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:20.208144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-25c8591f-dd1c-4728-9da0-9b5e41433f09 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:20.209418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-25c8591f-dd1c-4728-9da0-9b5e41433f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:20.217784 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1077/2172] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:20.222104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd1791-fe60-4834-8ba0-4d069926f9cd None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:20.231613 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd1791-fe60-4834-8ba0-4d069926f9cd None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:20.236729 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524675ee-42d4-45cf-bc15-8102e8d9a97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:20.237333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524675ee-42d4-45cf-bc15-8102e8d9a97f admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:20.237614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524675ee-42d4-45cf-bc15-8102e8d9a97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:20.237888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524675ee-42d4-45cf-bc15-8102e8d9a97f admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:20.238809 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524675ee-42d4-45cf-bc15-8102e8d9a97f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:50:20.238809 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524675ee-42d4-45cf-bc15-8102e8d9a97f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:20.240445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524675ee-42d4-45cf-bc15-8102e8d9a97f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:20.252380 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1096/2173] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:50:20.256631 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8591f-dd1c-4728-9da0-9b5e41433f09 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:20.265843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8591f-dd1c-4728-9da0-9b5e41433f09 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:20.271035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba769e-eac2-4e24-9ae6-b1647a2dc5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:20.271642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba769e-eac2-4e24-9ae6-b1647a2dc5d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:20.274900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba769e-eac2-4e24-9ae6-b1647a2dc5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:20.274900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba769e-eac2-4e24-9ae6-b1647a2dc5d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:20.274900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba769e-eac2-4e24-9ae6-b1647a2dc5d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:20.274900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba769e-eac2-4e24-9ae6-b1647a2dc5d3 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:20.446675 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1078/2174] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:50:20 2022] POST /identity/v3/projects => generated 407 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:20.451766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524675ee-42d4-45cf-bc15-8102e8d9a97f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:20.461779 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524675ee-42d4-45cf-bc15-8102e8d9a97f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:20.487453 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:20.519664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226da826-3794-4d2e-8054-ba4a9cd87fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:20.519664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226da826-3794-4d2e-8054-ba4a9cd87fea admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:20.519664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226da826-3794-4d2e-8054-ba4a9cd87fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:20.520037 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226da826-3794-4d2e-8054-ba4a9cd87fea admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:20.520523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226da826-3794-4d2e-8054-ba4a9cd87fea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:20.522482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226da826-3794-4d2e-8054-ba4a9cd87fea admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:20.603349 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1097/2175] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:50:20 2022] POST /identity/v3/users => generated 304 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:20.607190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba769e-eac2-4e24-9ae6-b1647a2dc5d3 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:20.617265 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba769e-eac2-4e24-9ae6-b1647a2dc5d3 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:20.622061 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98917467-35fd-4fc1-b1b7-7bb7b5ab01a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:20.622658 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98917467-35fd-4fc1-b1b7-7bb7b5ab01a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:20.622932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98917467-35fd-4fc1-b1b7-7bb7b5ab01a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:20.623201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98917467-35fd-4fc1-b1b7-7bb7b5ab01a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:20.623892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98917467-35fd-4fc1-b1b7-7bb7b5ab01a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:20.625481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98917467-35fd-4fc1-b1b7-7bb7b5ab01a7 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:20.635895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1079/2176] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:50:20.640661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226da826-3794-4d2e-8054-ba4a9cd87fea admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:20.650667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226da826-3794-4d2e-8054-ba4a9cd87fea admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:20.656078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ffd1e-67dc-4114-9fd1-a3b17f06d153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:20.656723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ffd1e-67dc-4114-9fd1-a3b17f06d153 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:20.657064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ffd1e-67dc-4114-9fd1-a3b17f06d153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:20.657344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ffd1e-67dc-4114-9fd1-a3b17f06d153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:20.657998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ffd1e-67dc-4114-9fd1-a3b17f06d153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:20.659663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ffd1e-67dc-4114-9fd1-a3b17f06d153 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:20.670219 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1098/2177] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:20.674342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98917467-35fd-4fc1-b1b7-7bb7b5ab01a7 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:20.684106 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98917467-35fd-4fc1-b1b7-7bb7b5ab01a7 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:20.689153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4daa9b-fe6f-41ea-a1d1-9d458522d7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:20.689795 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4daa9b-fe6f-41ea-a1d1-9d458522d7c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:20.690068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4daa9b-fe6f-41ea-a1d1-9d458522d7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:20.690336 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4daa9b-fe6f-41ea-a1d1-9d458522d7c8 admin admin] PATH_INFO: `/v3/projects/7b46ee5a2b8a49f58d257aae0746fce5/users/1085ee8b6d16490dbf168d8edf6bba1c/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:20.704958 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4daa9b-fe6f-41ea-a1d1-9d458522d7c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b46ee5a2b8a49f58d257aae0746fce5, user_id=1085ee8b6d16490dbf168d8edf6bba1c, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:20.707593 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4daa9b-fe6f-41ea-a1d1-9d458522d7c8 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:20.710563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:20.803493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1080/2178] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:20 2022] PUT /identity/v3/projects/7b46ee5a2b8a49f58d257aae0746fce5/users/1085ee8b6d16490dbf168d8edf6bba1c/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:20.815256 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d307aee-f687-46d7-965a-ce9ba2d035be None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:20.815544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d307aee-f687-46d7-965a-ce9ba2d035be None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:20.815832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d307aee-f687-46d7-965a-ce9ba2d035be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:20.844951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0d307aee-f687-46d7-965a-ce9ba2d035be None None] MFA Rules not processed for user `1085ee8b6d16490dbf168d8edf6bba1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:20.846175 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0d307aee-f687-46d7-965a-ce9ba2d035be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:20.914144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1099/2179] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:20 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:20.921634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4daa9b-fe6f-41ea-a1d1-9d458522d7c8 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:20.983904 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4daa9b-fe6f-41ea-a1d1-9d458522d7c8 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:20.988692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0eb099-d6cb-4cdb-93dc-437e83b1dd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:20.989514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0eb099-d6cb-4cdb-93dc-437e83b1dd20 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:20.989709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0eb099-d6cb-4cdb-93dc-437e83b1dd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:20.990039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0eb099-d6cb-4cdb-93dc-437e83b1dd20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:20.990732 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0eb099-d6cb-4cdb-93dc-437e83b1dd20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:20.992840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0eb099-d6cb-4cdb-93dc-437e83b1dd20 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:21.084883 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1081/2180] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:50:20 2022] POST /identity/v3/projects => generated 409 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:21.088759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d307aee-f687-46d7-965a-ce9ba2d035be None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:21.101039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d307aee-f687-46d7-965a-ce9ba2d035be None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:21.170737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffbdd4-e4e1-4482-bfbe-b9ecffbb410e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:21.171350 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffbdd4-e4e1-4482-bfbe-b9ecffbb410e admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:21.171629 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffbdd4-e4e1-4482-bfbe-b9ecffbb410e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:21.171895 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffbdd4-e4e1-4482-bfbe-b9ecffbb410e admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:21.172613 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffbdd4-e4e1-4482-bfbe-b9ecffbb410e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:21.174529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffbdd4-e4e1-4482-bfbe-b9ecffbb410e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:21.274007 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0eb099-d6cb-4cdb-93dc-437e83b1dd20 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:21.275766 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-bd0eb099-d6cb-4cdb-93dc-437e83b1dd20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:21.283465 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1100/2181] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:50:21 2022] POST /identity/v3/users => generated 311 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:50:21.287772 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffbdd4-e4e1-4482-bfbe-b9ecffbb410e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:21.292349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0eb099-d6cb-4cdb-93dc-437e83b1dd20 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:21.299564 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e857f34-1119-4502-bf97-c95ee1e3bcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:21.299719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffbdd4-e4e1-4482-bfbe-b9ecffbb410e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:21.300293 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e857f34-1119-4502-bf97-c95ee1e3bcb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:21.300611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e857f34-1119-4502-bf97-c95ee1e3bcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:21.301038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e857f34-1119-4502-bf97-c95ee1e3bcb4 admin admin] PATH_INFO: `/v3/users/c480e0a5f049480b9f1ed0f877a25b83` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:21.302289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e857f34-1119-4502-bf97-c95ee1e3bcb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c480e0a5f049480b9f1ed0f877a25b83)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:21.304252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e857f34-1119-4502-bf97-c95ee1e3bcb4 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:21.305601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5b6b3-281b-4aaa-bd14-4954a2d6b5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:21.306483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5b6b3-281b-4aaa-bd14-4954a2d6b5bd admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:21.306837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5b6b3-281b-4aaa-bd14-4954a2d6b5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:21.307206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5b6b3-281b-4aaa-bd14-4954a2d6b5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:21.307925 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5b6b3-281b-4aaa-bd14-4954a2d6b5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:21.309643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5b6b3-281b-4aaa-bd14-4954a2d6b5bd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:21.327998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1101/2182] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:21.332329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5b6b3-281b-4aaa-bd14-4954a2d6b5bd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:21.343820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5b6b3-281b-4aaa-bd14-4954a2d6b5bd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:21.348991 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806c163-14ec-4c44-8ee0-dc8372705803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:21.349773 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806c163-14ec-4c44-8ee0-dc8372705803 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:21.350045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806c163-14ec-4c44-8ee0-dc8372705803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:21.350323 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806c163-14ec-4c44-8ee0-dc8372705803 admin admin] PATH_INFO: `/v3/projects/e0707a52038a4974b8d64cf0ec7a761f/users/1b6c6195302d4b0d9354730e8e4f001e/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:21.363420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:21.366878 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806c163-14ec-4c44-8ee0-dc8372705803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0707a52038a4974b8d64cf0ec7a761f, user_id=1b6c6195302d4b0d9354730e8e4f001e, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:21.369666 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806c163-14ec-4c44-8ee0-dc8372705803 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:21.491134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1102/2183] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:21 2022] PUT /identity/v3/projects/e0707a52038a4974b8d64cf0ec7a761f/users/1b6c6195302d4b0d9354730e8e4f001e/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:50:21.494963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806c163-14ec-4c44-8ee0-dc8372705803 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:21.528378 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2e857f34-1119-4502-bf97-c95ee1e3bcb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c480e0a5f049480b9f1ed0f877a25b83'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:21.529099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2e857f34-1119-4502-bf97-c95ee1e3bcb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c480e0a5f049480b9f1ed0f877a25b83'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:21.530191 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2e857f34-1119-4502-bf97-c95ee1e3bcb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c480e0a5f049480b9f1ed0f877a25b83'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:21.537342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2e857f34-1119-4502-bf97-c95ee1e3bcb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c480e0a5f049480b9f1ed0f877a25b83'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:21.558961 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806c163-14ec-4c44-8ee0-dc8372705803 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:21.563544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f5e45-93cb-4ffe-8162-7300b63b6dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:21.564152 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f5e45-93cb-4ffe-8162-7300b63b6dae admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:21.564462 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f5e45-93cb-4ffe-8162-7300b63b6dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:21.564739 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f5e45-93cb-4ffe-8162-7300b63b6dae admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:21.565388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f5e45-93cb-4ffe-8162-7300b63b6dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:21.567005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f5e45-93cb-4ffe-8162-7300b63b6dae admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:21.576893 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1103/2184] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:50:21.581174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f5e45-93cb-4ffe-8162-7300b63b6dae admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:21.582996 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3c4f5e45-93cb-4ffe-8162-7300b63b6dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:21.595819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f5e45-93cb-4ffe-8162-7300b63b6dae admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:21.600461 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78b852-d4a4-4aa3-a79a-a1f35b777701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:21.601109 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78b852-d4a4-4aa3-a79a-a1f35b777701 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:21.601391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78b852-d4a4-4aa3-a79a-a1f35b777701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:21.601665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78b852-d4a4-4aa3-a79a-a1f35b777701 admin admin] PATH_INFO: `/v3/projects/e0707a52038a4974b8d64cf0ec7a761f/users/1b6c6195302d4b0d9354730e8e4f001e/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:21.603555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78b852-d4a4-4aa3-a79a-a1f35b777701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0707a52038a4974b8d64cf0ec7a761f, user_id=1b6c6195302d4b0d9354730e8e4f001e, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:21.606312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78b852-d4a4-4aa3-a79a-a1f35b777701 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:21.608732 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2e857f34-1119-4502-bf97-c95ee1e3bcb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c480e0a5f049480b9f1ed0f877a25b83'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:21.616902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1104/2185] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:21 2022] PUT /identity/v3/projects/e0707a52038a4974b8d64cf0ec7a761f/users/1b6c6195302d4b0d9354730e8e4f001e/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:50:21.627516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a57960-bb21-4d4b-ac27-fad61c1a603f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:21.627808 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a57960-bb21-4d4b-ac27-fad61c1a603f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:21.628081 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a57960-bb21-4d4b-ac27-fad61c1a603f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:21.628589 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1082/2186] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:21 2022] DELETE /identity/v3/users/c480e0a5f049480b9f1ed0f877a25b83 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:50:21.632861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da271d0f-cc33-4c6e-b0ca-dd159f3c0915 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:21.633145 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da271d0f-cc33-4c6e-b0ca-dd159f3c0915 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:21.633415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da271d0f-cc33-4c6e-b0ca-dd159f3c0915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:21.651906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-da271d0f-cc33-4c6e-b0ca-dd159f3c0915 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:21.653133 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-da271d0f-cc33-4c6e-b0ca-dd159f3c0915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:21.656218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e7a57960-bb21-4d4b-ac27-fad61c1a603f None None] MFA Rules not processed for user `1b6c6195302d4b0d9354730e8e4f001e`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:21.657429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e7a57960-bb21-4d4b-ac27-fad61c1a603f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:21.702542 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:21.715516 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1083/2187] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:21.725825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da271d0f-cc33-4c6e-b0ca-dd159f3c0915 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:21.727176 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-da271d0f-cc33-4c6e-b0ca-dd159f3c0915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:21.734232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1105/2188] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:21 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:21.745063 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a57960-bb21-4d4b-ac27-fad61c1a603f None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:21.746674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e7a57960-bb21-4d4b-ac27-fad61c1a603f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:21.784226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da271d0f-cc33-4c6e-b0ca-dd159f3c0915 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:21.788570 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03dcbd-0f7c-441a-a385-8959e4b61728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:21.789247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03dcbd-0f7c-441a-a385-8959e4b61728 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:21.789535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03dcbd-0f7c-441a-a385-8959e4b61728 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:21.789809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03dcbd-0f7c-441a-a385-8959e4b61728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:21.799652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03dcbd-0f7c-441a-a385-8959e4b61728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:21.801102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03dcbd-0f7c-441a-a385-8959e4b61728 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:21.804707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a57960-bb21-4d4b-ac27-fad61c1a603f None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:21.809474 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8808fc-babf-4be9-993a-dd1f870f088d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:21.810139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8808fc-babf-4be9-993a-dd1f870f088d service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:21.810350 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8808fc-babf-4be9-993a-dd1f870f088d service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:21.811167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8808fc-babf-4be9-993a-dd1f870f088d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:21.811744 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1084/2189] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:50:21.821202 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8808fc-babf-4be9-993a-dd1f870f088d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:21.822633 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8808fc-babf-4be9-993a-dd1f870f088d service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:21.832437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1106/2190] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:21 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:50:22.236423 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03dcbd-0f7c-441a-a385-8959e4b61728 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:22.237895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ba03dcbd-0f7c-441a-a385-8959e4b61728 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:22.251137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03dcbd-0f7c-441a-a385-8959e4b61728 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:22.256202 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c36ef-207e-4dd4-80c4-624cffdaa203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:22.256665 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c36ef-207e-4dd4-80c4-624cffdaa203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:22.256970 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c36ef-207e-4dd4-80c4-624cffdaa203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:22.257237 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c36ef-207e-4dd4-80c4-624cffdaa203 admin admin] PATH_INFO: `/v3/projects/5e7c3f5fc1b3403da3f37a7e0afdc657` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:22.262737 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c36ef-207e-4dd4-80c4-624cffdaa203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e7c3f5fc1b3403da3f37a7e0afdc657)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:22.264778 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c36ef-207e-4dd4-80c4-624cffdaa203 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:22.283064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8808fc-babf-4be9-993a-dd1f870f088d service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:22.283064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5d8808fc-babf-4be9-993a-dd1f870f088d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:22.283064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:22.285622 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8808fc-babf-4be9-993a-dd1f870f088d service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:22.289532 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddb281-88a2-4b1a-bf09-f495f3382bc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:22.290139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddb281-88a2-4b1a-bf09-f495f3382bc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:22.290417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddb281-88a2-4b1a-bf09-f495f3382bc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:22.290691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddb281-88a2-4b1a-bf09-f495f3382bc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:22.292440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-30ddb281-88a2-4b1a-bf09-f495f3382bc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:22.360476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddb281-88a2-4b1a-bf09-f495f3382bc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:22.361843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddb281-88a2-4b1a-bf09-f495f3382bc1 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:22.370948 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1107/2191] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:22 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:50:22.432276 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-eb5c36ef-207e-4dd4-80c4-624cffdaa203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e7c3f5fc1b3403da3f37a7e0afdc657', 'request_id': 'req-eb5c36ef-207e-4dd4-80c4-624cffdaa203'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:22.438662 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-eb5c36ef-207e-4dd4-80c4-624cffdaa203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e7c3f5fc1b3403da3f37a7e0afdc657', 'request_id': 'req-eb5c36ef-207e-4dd4-80c4-624cffdaa203'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:22.444531 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-eb5c36ef-207e-4dd4-80c4-624cffdaa203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e7c3f5fc1b3403da3f37a7e0afdc657', 'request_id': 'req-eb5c36ef-207e-4dd4-80c4-624cffdaa203'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:22.590291 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-eb5c36ef-207e-4dd4-80c4-624cffdaa203 admin admin] The token cache is being invalidate because project 5e7c3f5fc1b3403da3f37a7e0afdc657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:22.590703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-eb5c36ef-207e-4dd4-80c4-624cffdaa203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:22.591417 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-eb5c36ef-207e-4dd4-80c4-624cffdaa203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:22.593799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1085/2192] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:22 2022] DELETE /identity/v3/projects/5e7c3f5fc1b3403da3f37a7e0afdc657 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:22.598413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddb281-88a2-4b1a-bf09-f495f3382bc1 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:22.609289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddb281-88a2-4b1a-bf09-f495f3382bc1 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:22.667897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efeecf-0af5-484f-8685-a781120f23a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:22.668601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efeecf-0af5-484f-8685-a781120f23a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:22.671372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efeecf-0af5-484f-8685-a781120f23a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:22.671372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efeecf-0af5-484f-8685-a781120f23a4 admin admin] PATH_INFO: `/v3/users/11292a99199f4eaaa46057b6a350520c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:22.671372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efeecf-0af5-484f-8685-a781120f23a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11292a99199f4eaaa46057b6a350520c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:22.672219 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efeecf-0af5-484f-8685-a781120f23a4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:22.865422 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-26efeecf-0af5-484f-8685-a781120f23a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11292a99199f4eaaa46057b6a350520c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:22.866073 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-26efeecf-0af5-484f-8685-a781120f23a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11292a99199f4eaaa46057b6a350520c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:22.866681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-26efeecf-0af5-484f-8685-a781120f23a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11292a99199f4eaaa46057b6a350520c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:22.942796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-26efeecf-0af5-484f-8685-a781120f23a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11292a99199f4eaaa46057b6a350520c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:22.954052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-26efeecf-0af5-484f-8685-a781120f23a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11292a99199f4eaaa46057b6a350520c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:22.974260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1108/2193] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:22 2022] DELETE /identity/v3/users/11292a99199f4eaaa46057b6a350520c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:23.416450 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c36ef-207e-4dd4-80c4-624cffdaa203 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:23.417914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-eb5c36ef-207e-4dd4-80c4-624cffdaa203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:23.432973 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c36ef-207e-4dd4-80c4-624cffdaa203 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:23.489871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ede0c-8fdc-4545-9601-126a2c426e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:23.490495 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ede0c-8fdc-4545-9601-126a2c426e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:23.490773 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ede0c-8fdc-4545-9601-126a2c426e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:23.491043 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ede0c-8fdc-4545-9601-126a2c426e4d admin admin] PATH_INFO: `/v3/projects/eeaf0dc6c1f84183976a4ab5b83d077e` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:23.492156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ede0c-8fdc-4545-9601-126a2c426e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeaf0dc6c1f84183976a4ab5b83d077e)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:23.494083 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ede0c-8fdc-4545-9601-126a2c426e4d admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:23.517494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:23.625587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad1ede0c-8fdc-4545-9601-126a2c426e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeaf0dc6c1f84183976a4ab5b83d077e', 'request_id': 'req-ad1ede0c-8fdc-4545-9601-126a2c426e4d'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:23.631996 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad1ede0c-8fdc-4545-9601-126a2c426e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeaf0dc6c1f84183976a4ab5b83d077e', 'request_id': 'req-ad1ede0c-8fdc-4545-9601-126a2c426e4d'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:23.637721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad1ede0c-8fdc-4545-9601-126a2c426e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeaf0dc6c1f84183976a4ab5b83d077e', 'request_id': 'req-ad1ede0c-8fdc-4545-9601-126a2c426e4d'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:23.733922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad1ede0c-8fdc-4545-9601-126a2c426e4d admin admin] The token cache is being invalidate because project eeaf0dc6c1f84183976a4ab5b83d077e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:23.734218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad1ede0c-8fdc-4545-9601-126a2c426e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:23.736179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad1ede0c-8fdc-4545-9601-126a2c426e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:23.736477 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1086/2194] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:23 2022] DELETE /identity/v3/projects/eeaf0dc6c1f84183976a4ab5b83d077e => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:23.740490 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efeecf-0af5-484f-8685-a781120f23a4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:23.742012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-26efeecf-0af5-484f-8685-a781120f23a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:23.759119 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efeecf-0af5-484f-8685-a781120f23a4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:23.775701 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:23.815060 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba32fb-7e25-4012-a185-13fcaa034d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:23.815696 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba32fb-7e25-4012-a185-13fcaa034d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:23.816321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba32fb-7e25-4012-a185-13fcaa034d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:23.816321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba32fb-7e25-4012-a185-13fcaa034d7e admin admin] PATH_INFO: `/v3/users/2b782eb5b6f94ae0a3284d745fe0c8ee` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:23.817453 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba32fb-7e25-4012-a185-13fcaa034d7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b782eb5b6f94ae0a3284d745fe0c8ee)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:23.819379 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba32fb-7e25-4012-a185-13fcaa034d7e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:24.038189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a5ba32fb-7e25-4012-a185-13fcaa034d7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b782eb5b6f94ae0a3284d745fe0c8ee'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:24.038887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a5ba32fb-7e25-4012-a185-13fcaa034d7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b782eb5b6f94ae0a3284d745fe0c8ee'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:24.039615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a5ba32fb-7e25-4012-a185-13fcaa034d7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b782eb5b6f94ae0a3284d745fe0c8ee'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:24.114069 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a5ba32fb-7e25-4012-a185-13fcaa034d7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b782eb5b6f94ae0a3284d745fe0c8ee'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:24.122179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a5ba32fb-7e25-4012-a185-13fcaa034d7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b782eb5b6f94ae0a3284d745fe0c8ee'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:24.141496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1109/2195] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:23 2022] DELETE /identity/v3/users/2b782eb5b6f94ae0a3284d745fe0c8ee => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:24.547316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ede0c-8fdc-4545-9601-126a2c426e4d admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:24.548820 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ad1ede0c-8fdc-4545-9601-126a2c426e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:24.561513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ede0c-8fdc-4545-9601-126a2c426e4d admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:24.618425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:24.619066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:24.619356 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:24.619630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5 admin admin] PATH_INFO: `/v3/projects/98bfff925dc540afa2e6feef3acdb79d` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:24.621060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98bfff925dc540afa2e6feef3acdb79d)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:24.622984 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:24.795830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98bfff925dc540afa2e6feef3acdb79d', 'request_id': 'req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:24.802633 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98bfff925dc540afa2e6feef3acdb79d', 'request_id': 'req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:24.809124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98bfff925dc540afa2e6feef3acdb79d', 'request_id': 'req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:24.904134 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5 admin admin] The token cache is being invalidate because project 98bfff925dc540afa2e6feef3acdb79d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:24.904443 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:24.905738 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:24.907256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1087/2196] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:24 2022] DELETE /identity/v3/projects/98bfff925dc540afa2e6feef3acdb79d => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:33.103489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedb8a9-9517-4b6b-9d9d-3b5a72960fac None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:33.103955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedb8a9-9517-4b6b-9d9d-3b5a72960fac None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:33.104046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedb8a9-9517-4b6b-9d9d-3b5a72960fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:33.125020 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4eedb8a9-9517-4b6b-9d9d-3b5a72960fac None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:33.126231 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4eedb8a9-9517-4b6b-9d9d-3b5a72960fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:33.182079 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1110/2197] 10.222.0.70 () {58 vars in 1009 bytes} [Wed Aug 24 16:50:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:33.224937 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:33.226350 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:33.270586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:33.296160 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6ea8d-728e-4e2d-ba44-d6e529ef65c5 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:33.301842 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f08be-bec5-46d6-be91-d0ba04ea5073 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:33.302451 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f08be-bec5-46d6-be91-d0ba04ea5073 service swift] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:33.302736 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f08be-bec5-46d6-be91-d0ba04ea5073 service swift] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:33.303010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f08be-bec5-46d6-be91-d0ba04ea5073 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:33.311161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f08be-bec5-46d6-be91-d0ba04ea5073 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:33.312781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f08be-bec5-46d6-be91-d0ba04ea5073 service swift] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:33.320845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1088/2198] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 16:50:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:50:42.946805 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedb8a9-9517-4b6b-9d9d-3b5a72960fac None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:42.952166 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4eedb8a9-9517-4b6b-9d9d-3b5a72960fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:43.024642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:43.049914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedb8a9-9517-4b6b-9d9d-3b5a72960fac None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:43.055671 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad401ec-1900-4f3d-bcd0-01363f27fbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:43.056538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad401ec-1900-4f3d-bcd0-01363f27fbdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:43.056971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad401ec-1900-4f3d-bcd0-01363f27fbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:43.057423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad401ec-1900-4f3d-bcd0-01363f27fbdb admin admin] PATH_INFO: `/v3/users/1085ee8b6d16490dbf168d8edf6bba1c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:43.058757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad401ec-1900-4f3d-bcd0-01363f27fbdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1085ee8b6d16490dbf168d8edf6bba1c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:43.060923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad401ec-1900-4f3d-bcd0-01363f27fbdb admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:43.288113 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-bad401ec-1900-4f3d-bcd0-01363f27fbdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1085ee8b6d16490dbf168d8edf6bba1c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:43.288770 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-bad401ec-1900-4f3d-bcd0-01363f27fbdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1085ee8b6d16490dbf168d8edf6bba1c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:43.289578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-bad401ec-1900-4f3d-bcd0-01363f27fbdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1085ee8b6d16490dbf168d8edf6bba1c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:43.359913 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=14 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:43.369711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-bad401ec-1900-4f3d-bcd0-01363f27fbdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1085ee8b6d16490dbf168d8edf6bba1c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:43.378794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-bad401ec-1900-4f3d-bcd0-01363f27fbdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1085ee8b6d16490dbf168d8edf6bba1c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:43.403249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1111/2199] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:42 2022] DELETE /identity/v3/users/1085ee8b6d16490dbf168d8edf6bba1c => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:43.409583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9d7d5-31af-4ff5-bea9-38eaaad893ff None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:43.410226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9d7d5-31af-4ff5-bea9-38eaaad893ff None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:43.410878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9d7d5-31af-4ff5-bea9-38eaaad893ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:43.444912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad401ec-1900-4f3d-bcd0-01363f27fbdb admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:43.444912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bad401ec-1900-4f3d-bcd0-01363f27fbdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:43.454850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-77a9d7d5-31af-4ff5-bea9-38eaaad893ff None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:43.457137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-77a9d7d5-31af-4ff5-bea9-38eaaad893ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:43.534169 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1089/2200] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:43.534991 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad401ec-1900-4f3d-bcd0-01363f27fbdb admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:43.541890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f865cf-cda0-4120-b1f4-3f8fc8aa8cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:43.541890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f865cf-cda0-4120-b1f4-3f8fc8aa8cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:43.541890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f865cf-cda0-4120-b1f4-3f8fc8aa8cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:43.541890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f865cf-cda0-4120-b1f4-3f8fc8aa8cf8 admin admin] PATH_INFO: `/v3/users/398b57c208c642f6bcbe551c1891e42a` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:43.545861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f865cf-cda0-4120-b1f4-3f8fc8aa8cf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=398b57c208c642f6bcbe551c1891e42a)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:43.545861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f865cf-cda0-4120-b1f4-3f8fc8aa8cf8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:43.546578 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9d7d5-31af-4ff5-bea9-38eaaad893ff None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:43.549454 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-77a9d7d5-31af-4ff5-bea9-38eaaad893ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:43.634049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9d7d5-31af-4ff5-bea9-38eaaad893ff None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:43.637835 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45d4c9-e23b-4bc0-b8bf-2687bb3608c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:43.638453 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45d4c9-e23b-4bc0-b8bf-2687bb3608c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:43.638731 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45d4c9-e23b-4bc0-b8bf-2687bb3608c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:43.639008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45d4c9-e23b-4bc0-b8bf-2687bb3608c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:43.652229 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45d4c9-e23b-4bc0-b8bf-2687bb3608c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:43.653739 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45d4c9-e23b-4bc0-b8bf-2687bb3608c4 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:43.665251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1090/2201] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:50:43.757720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-43f865cf-cda0-4120-b1f4-3f8fc8aa8cf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '398b57c208c642f6bcbe551c1891e42a'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:43.758527 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-43f865cf-cda0-4120-b1f4-3f8fc8aa8cf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '398b57c208c642f6bcbe551c1891e42a'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:43.759215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-43f865cf-cda0-4120-b1f4-3f8fc8aa8cf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '398b57c208c642f6bcbe551c1891e42a'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:43.834748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-43f865cf-cda0-4120-b1f4-3f8fc8aa8cf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '398b57c208c642f6bcbe551c1891e42a'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:43.847916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-43f865cf-cda0-4120-b1f4-3f8fc8aa8cf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '398b57c208c642f6bcbe551c1891e42a'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:43.851327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:43.877805 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1112/2202] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:43 2022] DELETE /identity/v3/users/398b57c208c642f6bcbe551c1891e42a => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:43.881958 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad473d-c515-436f-8c8f-3c018c8c5865 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:43.882250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad473d-c515-436f-8c8f-3c018c8c5865 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:43.882527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad473d-c515-436f-8c8f-3c018c8c5865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:43.903529 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-6cad473d-c515-436f-8c8f-3c018c8c5865 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:43.907413 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6cad473d-c515-436f-8c8f-3c018c8c5865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:43.979615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1091/2203] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:43.988995 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f865cf-cda0-4120-b1f4-3f8fc8aa8cf8 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:43.991555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-43f865cf-cda0-4120-b1f4-3f8fc8aa8cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:44.066331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f865cf-cda0-4120-b1f4-3f8fc8aa8cf8 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:44.071495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38628c6f-22e6-4561-845d-0cbfb63aac70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:44.072124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38628c6f-22e6-4561-845d-0cbfb63aac70 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:44.072428 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38628c6f-22e6-4561-845d-0cbfb63aac70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:44.072755 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38628c6f-22e6-4561-845d-0cbfb63aac70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:44.083864 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38628c6f-22e6-4561-845d-0cbfb63aac70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:44.085548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38628c6f-22e6-4561-845d-0cbfb63aac70 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:44.113288 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1113/2204] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:44.143396 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad473d-c515-436f-8c8f-3c018c8c5865 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:44.144951 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6cad473d-c515-436f-8c8f-3c018c8c5865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:44.163102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad473d-c515-436f-8c8f-3c018c8c5865 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:44.172962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:44.173671 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:44.173951 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:44.174228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd admin admin] PATH_INFO: `/v3/projects/7b46ee5a2b8a49f58d257aae0746fce5` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:44.175425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b46ee5a2b8a49f58d257aae0746fce5)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:44.177464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:44.320434 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:44.329775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b46ee5a2b8a49f58d257aae0746fce5', 'request_id': 'req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:44.337087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b46ee5a2b8a49f58d257aae0746fce5', 'request_id': 'req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:44.344330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b46ee5a2b8a49f58d257aae0746fce5', 'request_id': 'req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:44.454827 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd admin admin] The token cache is being invalidate because project 7b46ee5a2b8a49f58d257aae0746fce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:44.456344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:44.457252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:44.462617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1092/2205] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:44 2022] DELETE /identity/v3/projects/7b46ee5a2b8a49f58d257aae0746fce5 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:44.463563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38628c6f-22e6-4561-845d-0cbfb63aac70 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:44.481522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38628c6f-22e6-4561-845d-0cbfb63aac70 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:44.583593 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:44.585208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:44.605101 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d446d2-43e1-45b3-b174-b8f4c5a9b276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:44.605907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d446d2-43e1-45b3-b174-b8f4c5a9b276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:44.606271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d446d2-43e1-45b3-b174-b8f4c5a9b276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:44.606640 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d446d2-43e1-45b3-b174-b8f4c5a9b276 admin admin] PATH_INFO: `/v3/users/1b6c6195302d4b0d9354730e8e4f001e` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:44.608041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d446d2-43e1-45b3-b174-b8f4c5a9b276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b6c6195302d4b0d9354730e8e4f001e)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:44.621443 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d446d2-43e1-45b3-b174-b8f4c5a9b276 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:44.716827 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969cca5-0659-407e-92cf-1ecc2fd7ecfd admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:44.724627 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1dd8a9-f76d-40ed-904c-12200eb450b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:44.725303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1dd8a9-f76d-40ed-904c-12200eb450b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:44.725581 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1dd8a9-f76d-40ed-904c-12200eb450b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:44.725850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1dd8a9-f76d-40ed-904c-12200eb450b8 admin admin] PATH_INFO: `/v3/projects/5002b5e821e84c2fa6de04d96a318f7b` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:44.727153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1dd8a9-f76d-40ed-904c-12200eb450b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5002b5e821e84c2fa6de04d96a318f7b)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:44.729237 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1dd8a9-f76d-40ed-904c-12200eb450b8 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:44.852823 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-81d446d2-43e1-45b3-b174-b8f4c5a9b276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b6c6195302d4b0d9354730e8e4f001e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:44.853784 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-81d446d2-43e1-45b3-b174-b8f4c5a9b276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b6c6195302d4b0d9354730e8e4f001e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:44.854613 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-81d446d2-43e1-45b3-b174-b8f4c5a9b276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b6c6195302d4b0d9354730e8e4f001e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:44.879891 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fb1dd8a9-f76d-40ed-904c-12200eb450b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5002b5e821e84c2fa6de04d96a318f7b', 'request_id': 'req-fb1dd8a9-f76d-40ed-904c-12200eb450b8'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:44.889628 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fb1dd8a9-f76d-40ed-904c-12200eb450b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5002b5e821e84c2fa6de04d96a318f7b', 'request_id': 'req-fb1dd8a9-f76d-40ed-904c-12200eb450b8'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:44.896290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fb1dd8a9-f76d-40ed-904c-12200eb450b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5002b5e821e84c2fa6de04d96a318f7b', 'request_id': 'req-fb1dd8a9-f76d-40ed-904c-12200eb450b8'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:44.961027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-81d446d2-43e1-45b3-b174-b8f4c5a9b276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b6c6195302d4b0d9354730e8e4f001e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:44.970504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-81d446d2-43e1-45b3-b174-b8f4c5a9b276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b6c6195302d4b0d9354730e8e4f001e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:44.990386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1114/2206] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:44 2022] DELETE /identity/v3/users/1b6c6195302d4b0d9354730e8e4f001e => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:45.046540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fb1dd8a9-f76d-40ed-904c-12200eb450b8 admin admin] The token cache is being invalidate because project 5002b5e821e84c2fa6de04d96a318f7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:45.047023 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fb1dd8a9-f76d-40ed-904c-12200eb450b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:45.047729 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fb1dd8a9-f76d-40ed-904c-12200eb450b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:45.049623 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1093/2207] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:44 2022] DELETE /identity/v3/projects/5002b5e821e84c2fa6de04d96a318f7b => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:50:45.057550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e931a08-8bfb-42b5-ab8c-df0d04eae545 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:45.058000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e931a08-8bfb-42b5-ab8c-df0d04eae545 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:45.058378 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e931a08-8bfb-42b5-ab8c-df0d04eae545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:45.067974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:45.098548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0e931a08-8bfb-42b5-ab8c-df0d04eae545 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:45.101885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0e931a08-8bfb-42b5-ab8c-df0d04eae545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:45.166314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1115/2208] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:45.178425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f441db-bd42-44e3-a662-68f1ce6e8315 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:45.179106 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f441db-bd42-44e3-a662-68f1ce6e8315 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:45.179590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f441db-bd42-44e3-a662-68f1ce6e8315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:45.220494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-13f441db-bd42-44e3-a662-68f1ce6e8315 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:45.221787 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-13f441db-bd42-44e3-a662-68f1ce6e8315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:45.231468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1094/2209] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:45.235842 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e931a08-8bfb-42b5-ab8c-df0d04eae545 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:45.247568 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e931a08-8bfb-42b5-ab8c-df0d04eae545 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:45.250678 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab960228-ccb7-45af-ae89-bcef07f8b7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:45.251296 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab960228-ccb7-45af-ae89-bcef07f8b7ce admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:45.251578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab960228-ccb7-45af-ae89-bcef07f8b7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:45.251850 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab960228-ccb7-45af-ae89-bcef07f8b7ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:45.252547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab960228-ccb7-45af-ae89-bcef07f8b7ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:50:45.252856 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab960228-ccb7-45af-ae89-bcef07f8b7ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:45.254593 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab960228-ccb7-45af-ae89-bcef07f8b7ce admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:45.270710 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1116/2210] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:50:45.273699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f441db-bd42-44e3-a662-68f1ce6e8315 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:45.283859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f441db-bd42-44e3-a662-68f1ce6e8315 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:45.288944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4952e-637c-4859-8db5-c56d9fdfa765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:45.293102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4952e-637c-4859-8db5-c56d9fdfa765 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:45.293102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4952e-637c-4859-8db5-c56d9fdfa765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:45.293102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4952e-637c-4859-8db5-c56d9fdfa765 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:45.293102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4952e-637c-4859-8db5-c56d9fdfa765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:45.293102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4952e-637c-4859-8db5-c56d9fdfa765 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:45.420454 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1095/2211] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:50:45 2022] POST /identity/v3/projects => generated 369 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:50:45.423146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab960228-ccb7-45af-ae89-bcef07f8b7ce admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:45.435337 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab960228-ccb7-45af-ae89-bcef07f8b7ce admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:45.473348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4952e-637c-4859-8db5-c56d9fdfa765 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:45.475437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-edd4952e-637c-4859-8db5-c56d9fdfa765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:45.520012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf843913-399d-4c3e-93e3-f5b029469cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:45.520940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf843913-399d-4c3e-93e3-f5b029469cfa admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:45.521235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf843913-399d-4c3e-93e3-f5b029469cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:45.521516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf843913-399d-4c3e-93e3-f5b029469cfa admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:45.522209 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf843913-399d-4c3e-93e3-f5b029469cfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:45.524383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf843913-399d-4c3e-93e3-f5b029469cfa admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:45.531474 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:45.579082 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4952e-637c-4859-8db5-c56d9fdfa765 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:45.585672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eba6dd-cd5b-4448-aa26-00ab440ed679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:45.586653 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eba6dd-cd5b-4448-aa26-00ab440ed679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:45.587260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eba6dd-cd5b-4448-aa26-00ab440ed679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:45.587765 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eba6dd-cd5b-4448-aa26-00ab440ed679 admin admin] PATH_INFO: `/v3/projects/e0707a52038a4974b8d64cf0ec7a761f` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:45.589402 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eba6dd-cd5b-4448-aa26-00ab440ed679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0707a52038a4974b8d64cf0ec7a761f)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:45.592941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eba6dd-cd5b-4448-aa26-00ab440ed679 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:45.619055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1117/2212] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:50:45 2022] POST /identity/v3/users => generated 285 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:45.624332 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf843913-399d-4c3e-93e3-f5b029469cfa admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:45.636881 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf843913-399d-4c3e-93e3-f5b029469cfa admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:45.643058 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40983bf5-95b4-48cf-8a42-9712a4e76b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:45.643639 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40983bf5-95b4-48cf-8a42-9712a4e76b09 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:45.643923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40983bf5-95b4-48cf-8a42-9712a4e76b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:45.644238 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40983bf5-95b4-48cf-8a42-9712a4e76b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:45.645406 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40983bf5-95b4-48cf-8a42-9712a4e76b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:45.646976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40983bf5-95b4-48cf-8a42-9712a4e76b09 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:45.658606 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1118/2213] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:45.662910 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40983bf5-95b4-48cf-8a42-9712a4e76b09 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:45.673566 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40983bf5-95b4-48cf-8a42-9712a4e76b09 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:45.678360 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f2d80-dc8a-48b4-b381-029e7fb5f431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:45.678940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f2d80-dc8a-48b4-b381-029e7fb5f431 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:45.679225 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f2d80-dc8a-48b4-b381-029e7fb5f431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:45.679502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f2d80-dc8a-48b4-b381-029e7fb5f431 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:45.680154 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f2d80-dc8a-48b4-b381-029e7fb5f431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:45.681854 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f2d80-dc8a-48b4-b381-029e7fb5f431 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:45.697235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1119/2214] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:45.698124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f2d80-dc8a-48b4-b381-029e7fb5f431 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:45.709843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f2d80-dc8a-48b4-b381-029e7fb5f431 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:45.717466 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a20e6-d7b5-4e97-9a1d-4f02a5c560df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:45.718888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a20e6-d7b5-4e97-9a1d-4f02a5c560df admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:45.724807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a20e6-d7b5-4e97-9a1d-4f02a5c560df admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:45.724807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a20e6-d7b5-4e97-9a1d-4f02a5c560df admin admin] PATH_INFO: `/v3/projects/cb8be6859bb843cba4fe575dc3737e9b/users/a4de9173f677460ca0687dd5897cd134/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:45.738880 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a20e6-d7b5-4e97-9a1d-4f02a5c560df admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb8be6859bb843cba4fe575dc3737e9b, user_id=a4de9173f677460ca0687dd5897cd134, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:45.742135 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a20e6-d7b5-4e97-9a1d-4f02a5c560df admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:45.755109 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-52eba6dd-cd5b-4448-aa26-00ab440ed679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0707a52038a4974b8d64cf0ec7a761f', 'request_id': 'req-52eba6dd-cd5b-4448-aa26-00ab440ed679'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:45.763948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-52eba6dd-cd5b-4448-aa26-00ab440ed679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0707a52038a4974b8d64cf0ec7a761f', 'request_id': 'req-52eba6dd-cd5b-4448-aa26-00ab440ed679'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:45.783175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-52eba6dd-cd5b-4448-aa26-00ab440ed679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0707a52038a4974b8d64cf0ec7a761f', 'request_id': 'req-52eba6dd-cd5b-4448-aa26-00ab440ed679'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:45.823784 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1120/2215] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:45 2022] PUT /identity/v3/projects/cb8be6859bb843cba4fe575dc3737e9b/users/a4de9173f677460ca0687dd5897cd134/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:45.834692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34703dda-a3f7-456e-b500-b8fbad008de7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:45.835198 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34703dda-a3f7-456e-b500-b8fbad008de7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:45.835596 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34703dda-a3f7-456e-b500-b8fbad008de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:45.892969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-34703dda-a3f7-456e-b500-b8fbad008de7 None None] MFA Rules not processed for user `a4de9173f677460ca0687dd5897cd134`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:45.899430 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-34703dda-a3f7-456e-b500-b8fbad008de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:45.951957 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:45.976244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-52eba6dd-cd5b-4448-aa26-00ab440ed679 admin admin] The token cache is being invalidate because project e0707a52038a4974b8d64cf0ec7a761f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:45.976682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-52eba6dd-cd5b-4448-aa26-00ab440ed679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:45.977274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-52eba6dd-cd5b-4448-aa26-00ab440ed679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:45.979128 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1096/2216] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:45 2022] DELETE /identity/v3/projects/e0707a52038a4974b8d64cf0ec7a761f => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:45.985393 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421612e-3a27-4f0c-ba88-ed0f1b785c23 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:45.989724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421612e-3a27-4f0c-ba88-ed0f1b785c23 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:45.990243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421612e-3a27-4f0c-ba88-ed0f1b785c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:46.028783 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1121/2217] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:46.030055 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-f421612e-3a27-4f0c-ba88-ed0f1b785c23 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:46.031498 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-f421612e-3a27-4f0c-ba88-ed0f1b785c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:46.037554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34703dda-a3f7-456e-b500-b8fbad008de7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:46.039217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-34703dda-a3f7-456e-b500-b8fbad008de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:46.103410 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34703dda-a3f7-456e-b500-b8fbad008de7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:46.108280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a13b3-fbed-4c0f-9fda-ebfbb9bda8e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:46.109692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a13b3-fbed-4c0f-9fda-ebfbb9bda8e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:46.110146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a13b3-fbed-4c0f-9fda-ebfbb9bda8e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:46.110414 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1097/2218] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:46.110631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a13b3-fbed-4c0f-9fda-ebfbb9bda8e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:46.121442 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a13b3-fbed-4c0f-9fda-ebfbb9bda8e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:46.122479 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f8c9c-6edd-45af-b3fa-01780f2ac0cf None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:46.122842 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f8c9c-6edd-45af-b3fa-01780f2ac0cf None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:46.123119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f8c9c-6edd-45af-b3fa-01780f2ac0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:46.123385 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a13b3-fbed-4c0f-9fda-ebfbb9bda8e3 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:46.135853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1122/2219] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:50:46.147707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-b39f8c9c-6edd-45af-b3fa-01780f2ac0cf None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:46.149196 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b39f8c9c-6edd-45af-b3fa-01780f2ac0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:46.163203 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1098/2220] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:46.173158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f8c9c-6edd-45af-b3fa-01780f2ac0cf None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:46.173334 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a13b3-fbed-4c0f-9fda-ebfbb9bda8e3 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:46.174632 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b39f8c9c-6edd-45af-b3fa-01780f2ac0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:46.186753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a13b3-fbed-4c0f-9fda-ebfbb9bda8e3 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:46.191826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa7895-5b9c-446d-879c-f83a16e3dc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:46.197676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f8c9c-6edd-45af-b3fa-01780f2ac0cf None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:46.197865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa7895-5b9c-446d-879c-f83a16e3dc75 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:46.197865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa7895-5b9c-446d-879c-f83a16e3dc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:46.197865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa7895-5b9c-446d-879c-f83a16e3dc75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:46.197865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa7895-5b9c-446d-879c-f83a16e3dc75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:50:46.197865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa7895-5b9c-446d-879c-f83a16e3dc75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:46.197865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa7895-5b9c-446d-879c-f83a16e3dc75 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:46.199439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c201e6-d288-4379-9d6b-1c34d04f33c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:46.200109 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c201e6-d288-4379-9d6b-1c34d04f33c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:46.200448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c201e6-d288-4379-9d6b-1c34d04f33c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:46.200730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c201e6-d288-4379-9d6b-1c34d04f33c9 admin admin] PATH_INFO: `/v3/users/a4de9173f677460ca0687dd5897cd134` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:46.201900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c201e6-d288-4379-9d6b-1c34d04f33c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4de9173f677460ca0687dd5897cd134)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:46.204161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c201e6-d288-4379-9d6b-1c34d04f33c9 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:46.213130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1123/2221] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:50:46.220355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa7895-5b9c-446d-879c-f83a16e3dc75 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:46.234714 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa7895-5b9c-446d-879c-f83a16e3dc75 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:46.241055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c6d6d-b690-455f-b1f7-2b8efdaf549d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:46.242960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c6d6d-b690-455f-b1f7-2b8efdaf549d admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:46.243465 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c6d6d-b690-455f-b1f7-2b8efdaf549d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:46.244273 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c6d6d-b690-455f-b1f7-2b8efdaf549d admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:46.247050 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c6d6d-b690-455f-b1f7-2b8efdaf549d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:46.249141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c6d6d-b690-455f-b1f7-2b8efdaf549d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:46.396910 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1124/2222] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:50:46 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:46.397999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c6d6d-b690-455f-b1f7-2b8efdaf549d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:46.401553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-37c201e6-d288-4379-9d6b-1c34d04f33c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4de9173f677460ca0687dd5897cd134'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:46.402163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-37c201e6-d288-4379-9d6b-1c34d04f33c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4de9173f677460ca0687dd5897cd134'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:46.402779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-37c201e6-d288-4379-9d6b-1c34d04f33c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4de9173f677460ca0687dd5897cd134'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:46.404331 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:46.426416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-37c201e6-d288-4379-9d6b-1c34d04f33c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4de9173f677460ca0687dd5897cd134'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:46.494385 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c6d6d-b690-455f-b1f7-2b8efdaf549d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:46.501007 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f618c-9a32-4048-ae61-7759844aa467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:46.502354 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f618c-9a32-4048-ae61-7759844aa467 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:46.502643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f618c-9a32-4048-ae61-7759844aa467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:46.503040 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f618c-9a32-4048-ae61-7759844aa467 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:46.503605 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f618c-9a32-4048-ae61-7759844aa467 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:46.505843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f618c-9a32-4048-ae61-7759844aa467 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:46.545257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-37c201e6-d288-4379-9d6b-1c34d04f33c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4de9173f677460ca0687dd5897cd134'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:46.575105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1099/2223] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:46 2022] DELETE /identity/v3/users/a4de9173f677460ca0687dd5897cd134 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:46.580218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a39975-3344-458f-a83e-85415dfd2c03 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:46.580552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a39975-3344-458f-a83e-85415dfd2c03 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:46.580855 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a39975-3344-458f-a83e-85415dfd2c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:46.597413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1125/2224] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:50:46 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:46.601653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f618c-9a32-4048-ae61-7759844aa467 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:46.604718 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-985f618c-9a32-4048-ae61-7759844aa467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:46.607582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-63a39975-3344-458f-a83e-85415dfd2c03 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:46.608950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-63a39975-3344-458f-a83e-85415dfd2c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:46.631588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f618c-9a32-4048-ae61-7759844aa467 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:46.666447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:46.715472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf5662-7581-4253-b054-50d2d4154730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:46.716019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf5662-7581-4253-b054-50d2d4154730 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:46.716302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf5662-7581-4253-b054-50d2d4154730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:46.717180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf5662-7581-4253-b054-50d2d4154730 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:46.722250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1100/2225] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:46.722503 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf5662-7581-4253-b054-50d2d4154730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:46.722503 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf5662-7581-4253-b054-50d2d4154730 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:46.728357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a39975-3344-458f-a83e-85415dfd2c03 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:46.732148 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1126/2226] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:46.735599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf5662-7581-4253-b054-50d2d4154730 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:46.737982 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-63a39975-3344-458f-a83e-85415dfd2c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:46.749504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf5662-7581-4253-b054-50d2d4154730 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:46.772513 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e62d0-79c5-44a7-9e14-b2761857ce79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:46.774997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e62d0-79c5-44a7-9e14-b2761857ce79 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:46.777405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e62d0-79c5-44a7-9e14-b2761857ce79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:46.778457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e62d0-79c5-44a7-9e14-b2761857ce79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:46.779892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e62d0-79c5-44a7-9e14-b2761857ce79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:46.789377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e62d0-79c5-44a7-9e14-b2761857ce79 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:46.811020 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a39975-3344-458f-a83e-85415dfd2c03 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:46.815640 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1127/2227] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:50:46.816837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a95c59-d532-4068-b2b9-e240652bdf09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:46.819592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a95c59-d532-4068-b2b9-e240652bdf09 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:46.819592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a95c59-d532-4068-b2b9-e240652bdf09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:46.819592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a95c59-d532-4068-b2b9-e240652bdf09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:46.824594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e62d0-79c5-44a7-9e14-b2761857ce79 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:46.832418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a95c59-d532-4068-b2b9-e240652bdf09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:46.834154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a95c59-d532-4068-b2b9-e240652bdf09 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:46.850984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e62d0-79c5-44a7-9e14-b2761857ce79 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:46.865702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1101/2228] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:46.870618 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca40e6-7e60-4e7f-be08-e2962448f88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:46.873806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca40e6-7e60-4e7f-be08-e2962448f88f admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:46.873806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca40e6-7e60-4e7f-be08-e2962448f88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:46.876501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca40e6-7e60-4e7f-be08-e2962448f88f admin admin] PATH_INFO: `/v3/projects/2ada5631ac34434d91f15c0bc5489c44/users/507b3811edb540ea8c054b13bbe4ef05/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:46.893432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca40e6-7e60-4e7f-be08-e2962448f88f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ada5631ac34434d91f15c0bc5489c44, user_id=507b3811edb540ea8c054b13bbe4ef05, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:46.896257 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca40e6-7e60-4e7f-be08-e2962448f88f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:46.996491 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1128/2229] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:46 2022] PUT /identity/v3/projects/2ada5631ac34434d91f15c0bc5489c44/users/507b3811edb540ea8c054b13bbe4ef05/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:47.010634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d973f-2b34-4746-9ec0-4885edd8f901 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:47.010634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d973f-2b34-4746-9ec0-4885edd8f901 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:47.010634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d973f-2b34-4746-9ec0-4885edd8f901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:47.040838 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-bc6d973f-2b34-4746-9ec0-4885edd8f901 None None] MFA Rules not processed for user `507b3811edb540ea8c054b13bbe4ef05`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:47.042053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-bc6d973f-2b34-4746-9ec0-4885edd8f901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:47.126194 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:47.133327 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1102/2230] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:47.140811 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca40e6-7e60-4e7f-be08-e2962448f88f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:47.262435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca40e6-7e60-4e7f-be08-e2962448f88f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:47.272611 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6cdeb-734c-4214-832e-d31312852d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:47.273241 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6cdeb-734c-4214-832e-d31312852d03 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:47.273520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6cdeb-734c-4214-832e-d31312852d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:47.273813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6cdeb-734c-4214-832e-d31312852d03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:47.274459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6cdeb-734c-4214-832e-d31312852d03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:47.276367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6cdeb-734c-4214-832e-d31312852d03 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:47.391084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1129/2231] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:50:47 2022] POST /identity/v3/projects => generated 379 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:47.396784 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d973f-2b34-4746-9ec0-4885edd8f901 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:47.409576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d973f-2b34-4746-9ec0-4885edd8f901 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:47.430090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6cdeb-734c-4214-832e-d31312852d03 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:47.431920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e1c6cdeb-734c-4214-832e-d31312852d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:47.519865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb95680-63cc-4cb9-8ebb-97366ea9a3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:47.521132 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb95680-63cc-4cb9-8ebb-97366ea9a3e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:47.521598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb95680-63cc-4cb9-8ebb-97366ea9a3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:47.522080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb95680-63cc-4cb9-8ebb-97366ea9a3e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:47.522799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb95680-63cc-4cb9-8ebb-97366ea9a3e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:47.525257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb95680-63cc-4cb9-8ebb-97366ea9a3e8 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:47.525942 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6cdeb-734c-4214-832e-d31312852d03 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:47.532639 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4213b5-4c6c-4d60-b0e2-c85109b2de78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:47.533580 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4213b5-4c6c-4d60-b0e2-c85109b2de78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:47.534050 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4213b5-4c6c-4d60-b0e2-c85109b2de78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:47.534476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4213b5-4c6c-4d60-b0e2-c85109b2de78 admin admin] PATH_INFO: `/v3/projects/cb8be6859bb843cba4fe575dc3737e9b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:47.535995 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4213b5-4c6c-4d60-b0e2-c85109b2de78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb8be6859bb843cba4fe575dc3737e9b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:47.538359 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4213b5-4c6c-4d60-b0e2-c85109b2de78 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:47.550707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:47.627848 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1103/2232] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:50:47 2022] POST /identity/v3/users => generated 290 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:47.630002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb95680-63cc-4cb9-8ebb-97366ea9a3e8 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:47.643836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb95680-63cc-4cb9-8ebb-97366ea9a3e8 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:47.662308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5e4213b5-4c6c-4d60-b0e2-c85109b2de78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb8be6859bb843cba4fe575dc3737e9b', 'request_id': 'req-5e4213b5-4c6c-4d60-b0e2-c85109b2de78'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:47.670271 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5e4213b5-4c6c-4d60-b0e2-c85109b2de78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb8be6859bb843cba4fe575dc3737e9b', 'request_id': 'req-5e4213b5-4c6c-4d60-b0e2-c85109b2de78'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:47.681610 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5e4213b5-4c6c-4d60-b0e2-c85109b2de78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb8be6859bb843cba4fe575dc3737e9b', 'request_id': 'req-5e4213b5-4c6c-4d60-b0e2-c85109b2de78'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:47.725290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76320144-5374-4a27-9746-34f92146ea46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:47.726026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76320144-5374-4a27-9746-34f92146ea46 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:47.726306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76320144-5374-4a27-9746-34f92146ea46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:47.726603 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76320144-5374-4a27-9746-34f92146ea46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:47.727328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76320144-5374-4a27-9746-34f92146ea46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:47.729456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76320144-5374-4a27-9746-34f92146ea46 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:47.744020 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1104/2233] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:47.747104 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76320144-5374-4a27-9746-34f92146ea46 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:47.755083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5e4213b5-4c6c-4d60-b0e2-c85109b2de78 admin admin] The token cache is being invalidate because project cb8be6859bb843cba4fe575dc3737e9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:47.755372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5e4213b5-4c6c-4d60-b0e2-c85109b2de78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:47.756086 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5e4213b5-4c6c-4d60-b0e2-c85109b2de78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:47.757640 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1130/2234] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:47 2022] DELETE /identity/v3/projects/cb8be6859bb843cba4fe575dc3737e9b => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:50:47.763293 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbc10b-b22e-484d-9cd6-1c9994ccb542 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:47.763530 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbc10b-b22e-484d-9cd6-1c9994ccb542 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:47.763878 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbc10b-b22e-484d-9cd6-1c9994ccb542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:47.766310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76320144-5374-4a27-9746-34f92146ea46 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:47.783626 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee83f38-a756-4d78-8648-a657fab67aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:47.784286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee83f38-a756-4d78-8648-a657fab67aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:47.785224 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee83f38-a756-4d78-8648-a657fab67aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:47.785437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee83f38-a756-4d78-8648-a657fab67aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:47.786088 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee83f38-a756-4d78-8648-a657fab67aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:47.787725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee83f38-a756-4d78-8648-a657fab67aa3 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:47.790621 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-d4cbc10b-b22e-484d-9cd6-1c9994ccb542 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:47.790621 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d4cbc10b-b22e-484d-9cd6-1c9994ccb542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:47.800442 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1131/2235] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:47.802167 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1105/2236] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:50:47.808702 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbc10b-b22e-484d-9cd6-1c9994ccb542 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:47.810612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca690e-832c-4da6-a118-822b41ddcaca None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:47.811179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca690e-832c-4da6-a118-822b41ddcaca None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:47.812572 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d4cbc10b-b22e-484d-9cd6-1c9994ccb542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:47.816034 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca690e-832c-4da6-a118-822b41ddcaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:47.826605 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbc10b-b22e-484d-9cd6-1c9994ccb542 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:47.832603 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5ca90-aaa6-4024-a568-3cbf4ae65123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:47.833494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5ca90-aaa6-4024-a568-3cbf4ae65123 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:47.834140 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5ca90-aaa6-4024-a568-3cbf4ae65123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:47.834658 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5ca90-aaa6-4024-a568-3cbf4ae65123 admin admin] PATH_INFO: `/v3/projects/b2d9509e669a41cca873297a8131b94d/users/edd9277d9d744222b9b8fbc74e97dd41/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:47.837201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-e1ca690e-832c-4da6-a118-822b41ddcaca None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:47.838479 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e1ca690e-832c-4da6-a118-822b41ddcaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:47.846913 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1106/2237] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:47.851234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca690e-832c-4da6-a118-822b41ddcaca None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:47.861140 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca690e-832c-4da6-a118-822b41ddcaca None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:47.861919 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5ca90-aaa6-4024-a568-3cbf4ae65123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2d9509e669a41cca873297a8131b94d, user_id=edd9277d9d744222b9b8fbc74e97dd41, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:47.864814 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5ca90-aaa6-4024-a568-3cbf4ae65123 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:47.866415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95e202-68d3-4e4f-97a4-295e52f739fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:47.867227 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95e202-68d3-4e4f-97a4-295e52f739fe admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:47.867758 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95e202-68d3-4e4f-97a4-295e52f739fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:47.868205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95e202-68d3-4e4f-97a4-295e52f739fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:47.868963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95e202-68d3-4e4f-97a4-295e52f739fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:50:47.869270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95e202-68d3-4e4f-97a4-295e52f739fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:47.870874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95e202-68d3-4e4f-97a4-295e52f739fe admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:47.884650 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1107/2238] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:50:47.889200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95e202-68d3-4e4f-97a4-295e52f739fe admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:47.901589 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95e202-68d3-4e4f-97a4-295e52f739fe admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:47.906620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08773fdc-d746-4cc3-9572-fd3c1931856c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:47.907268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08773fdc-d746-4cc3-9572-fd3c1931856c admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:47.907549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08773fdc-d746-4cc3-9572-fd3c1931856c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:47.908050 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08773fdc-d746-4cc3-9572-fd3c1931856c admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:47.908714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08773fdc-d746-4cc3-9572-fd3c1931856c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:47.910812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08773fdc-d746-4cc3-9572-fd3c1931856c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:47.943574 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1132/2239] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:47 2022] PUT /identity/v3/projects/b2d9509e669a41cca873297a8131b94d/users/edd9277d9d744222b9b8fbc74e97dd41/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:47.955012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4def6-ba76-4e42-8e91-57f62e3b962e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:47.955391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4def6-ba76-4e42-8e91-57f62e3b962e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:47.955772 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4def6-ba76-4e42-8e91-57f62e3b962e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:47.994574 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a4e4def6-ba76-4e42-8e91-57f62e3b962e None None] MFA Rules not processed for user `edd9277d9d744222b9b8fbc74e97dd41`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:47.996731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a4e4def6-ba76-4e42-8e91-57f62e3b962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:48.038674 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1108/2240] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:50:47 2022] POST /identity/v3/projects => generated 399 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:48.043064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08773fdc-d746-4cc3-9572-fd3c1931856c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:48.081268 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1133/2241] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:48.086674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4def6-ba76-4e42-8e91-57f62e3b962e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:48.089379 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:48.144233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08773fdc-d746-4cc3-9572-fd3c1931856c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:48.162284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a8151-ecd8-44a3-a683-a1aa22868646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:48.162921 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a8151-ecd8-44a3-a683-a1aa22868646 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:48.163200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a8151-ecd8-44a3-a683-a1aa22868646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:48.163473 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a8151-ecd8-44a3-a683-a1aa22868646 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:48.164104 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a8151-ecd8-44a3-a683-a1aa22868646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:48.166074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a8151-ecd8-44a3-a683-a1aa22868646 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:48.166612 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:48.174740 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4def6-ba76-4e42-8e91-57f62e3b962e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:48.180682 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae88583-f3f3-491a-bcae-389234c642dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:48.181582 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae88583-f3f3-491a-bcae-389234c642dd admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:48.181687 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae88583-f3f3-491a-bcae-389234c642dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:48.182031 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae88583-f3f3-491a-bcae-389234c642dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:48.182674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae88583-f3f3-491a-bcae-389234c642dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:48.184658 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae88583-f3f3-491a-bcae-389234c642dd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:48.301256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1109/2242] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:50:48 2022] POST /identity/v3/users => generated 300 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:48.301871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a8151-ecd8-44a3-a683-a1aa22868646 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:48.322799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a8151-ecd8-44a3-a683-a1aa22868646 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:48.331575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e75074-6ea0-48a9-b33d-7358e07d2ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:48.332693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e75074-6ea0-48a9-b33d-7358e07d2ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:48.332978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e75074-6ea0-48a9-b33d-7358e07d2ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:48.333251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e75074-6ea0-48a9-b33d-7358e07d2ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:48.334016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e75074-6ea0-48a9-b33d-7358e07d2ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:48.334357 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1134/2243] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:50:48 2022] POST /identity/v3/projects => generated 377 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:48.338246 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae88583-f3f3-491a-bcae-389234c642dd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:48.340273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e75074-6ea0-48a9-b33d-7358e07d2ad7 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:48.357371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1110/2244] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:50:48.360749 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e75074-6ea0-48a9-b33d-7358e07d2ad7 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:48.393273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e75074-6ea0-48a9-b33d-7358e07d2ad7 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:48.462652 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae88583-f3f3-491a-bcae-389234c642dd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:48.471532 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abfa995-9be6-4765-b4fd-78935f999e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:48.472153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfa995-9be6-4765-b4fd-78935f999e49 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:48.472455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfa995-9be6-4765-b4fd-78935f999e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:48.472867 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfa995-9be6-4765-b4fd-78935f999e49 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:48.474540 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abfa995-9be6-4765-b4fd-78935f999e49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:48.476555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abfa995-9be6-4765-b4fd-78935f999e49 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:48.494061 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3cc566-6e3a-4ef7-8fdb-80bdf7c3d453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:48.494998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cc566-6e3a-4ef7-8fdb-80bdf7c3d453 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:48.495350 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cc566-6e3a-4ef7-8fdb-80bdf7c3d453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:48.495629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cc566-6e3a-4ef7-8fdb-80bdf7c3d453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:48.496708 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3cc566-6e3a-4ef7-8fdb-80bdf7c3d453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:48.502175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3cc566-6e3a-4ef7-8fdb-80bdf7c3d453 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:48.511878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1111/2245] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:48.516044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3cc566-6e3a-4ef7-8fdb-80bdf7c3d453 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:48.531798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3cc566-6e3a-4ef7-8fdb-80bdf7c3d453 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:48.537120 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba3751-a3a6-4ece-a069-7d53f8fd8e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:48.540333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba3751-a3a6-4ece-a069-7d53f8fd8e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:48.540333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba3751-a3a6-4ece-a069-7d53f8fd8e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:48.540333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba3751-a3a6-4ece-a069-7d53f8fd8e4e admin admin] PATH_INFO: `/v3/projects/57333e5df7104c449effa0d6c975b78d/users/0b5fec844d8f4985aebb5c314f9d3cb5/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:48.573797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba3751-a3a6-4ece-a069-7d53f8fd8e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=57333e5df7104c449effa0d6c975b78d, user_id=0b5fec844d8f4985aebb5c314f9d3cb5, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:48.573797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba3751-a3a6-4ece-a069-7d53f8fd8e4e admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:48.581450 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1135/2246] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:50:48 2022] POST /identity/v3/users => generated 295 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:50:48.582914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abfa995-9be6-4765-b4fd-78935f999e49 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:48.594628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abfa995-9be6-4765-b4fd-78935f999e49 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:48.599529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50fcc8-6f64-4c17-9db6-5ed6d762d0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:48.602440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50fcc8-6f64-4c17-9db6-5ed6d762d0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:48.602440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50fcc8-6f64-4c17-9db6-5ed6d762d0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:48.602440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50fcc8-6f64-4c17-9db6-5ed6d762d0e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:48.602440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50fcc8-6f64-4c17-9db6-5ed6d762d0e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:48.603056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50fcc8-6f64-4c17-9db6-5ed6d762d0e4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:48.618360 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1136/2247] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:48.620938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50fcc8-6f64-4c17-9db6-5ed6d762d0e4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:48.640952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50fcc8-6f64-4c17-9db6-5ed6d762d0e4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:48.652906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1112/2248] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:48 2022] PUT /identity/v3/projects/57333e5df7104c449effa0d6c975b78d/users/0b5fec844d8f4985aebb5c314f9d3cb5/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:48.668450 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9017e1-bb51-445c-a8ef-db20979b2429 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:48.668450 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9017e1-bb51-445c-a8ef-db20979b2429 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:48.668450 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9017e1-bb51-445c-a8ef-db20979b2429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:48.716849 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-2b9017e1-bb51-445c-a8ef-db20979b2429 None None] MFA Rules not processed for user `0b5fec844d8f4985aebb5c314f9d3cb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:48.718903 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2b9017e1-bb51-445c-a8ef-db20979b2429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:48.725551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eece8c-d56f-4a49-8e2c-2d5a243fee69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:48.726237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eece8c-d56f-4a49-8e2c-2d5a243fee69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:48.726519 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eece8c-d56f-4a49-8e2c-2d5a243fee69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:48.726798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eece8c-d56f-4a49-8e2c-2d5a243fee69 admin admin] PATH_INFO: `/v3/projects/8cd84504623f4ac6aa8ef7aa9156d6e2/users/f7bdba840d204dac8ae03756fbfdea10/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:48.754731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eece8c-d56f-4a49-8e2c-2d5a243fee69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd84504623f4ac6aa8ef7aa9156d6e2, user_id=f7bdba840d204dac8ae03756fbfdea10, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:48.767278 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eece8c-d56f-4a49-8e2c-2d5a243fee69 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:48.825297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1113/2249] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:48 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:48.854695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9017e1-bb51-445c-a8ef-db20979b2429 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:48.861487 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2b9017e1-bb51-445c-a8ef-db20979b2429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:48.861608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1137/2250] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:48 2022] PUT /identity/v3/projects/8cd84504623f4ac6aa8ef7aa9156d6e2/users/f7bdba840d204dac8ae03756fbfdea10/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:50:48.863977 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eece8c-d56f-4a49-8e2c-2d5a243fee69 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:48.871015 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:48.910173 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:48.915456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9017e1-bb51-445c-a8ef-db20979b2429 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:48.925934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eece8c-d56f-4a49-8e2c-2d5a243fee69 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:48.930803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3de76-210d-4966-a208-1843843c1428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:48.941846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3de76-210d-4966-a208-1843843c1428 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:48.941846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3de76-210d-4966-a208-1843843c1428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:48.941846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3de76-210d-4966-a208-1843843c1428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:48.941846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3de76-210d-4966-a208-1843843c1428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:48.941846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3de76-210d-4966-a208-1843843c1428 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:48.943929 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1138/2251] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:50:48.947987 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3de76-210d-4966-a208-1843843c1428 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:48.958875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3de76-210d-4966-a208-1843843c1428 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:48.963969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae815b4-b1b3-4b14-ab09-98c5a0eca459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:48.965072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae815b4-b1b3-4b14-ab09-98c5a0eca459 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:48.965360 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae815b4-b1b3-4b14-ab09-98c5a0eca459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:48.965675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae815b4-b1b3-4b14-ab09-98c5a0eca459 admin admin] PATH_INFO: `/v3/projects/8cd84504623f4ac6aa8ef7aa9156d6e2/users/f7bdba840d204dac8ae03756fbfdea10/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:48.966537 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdd77e-66bd-43cf-9cbc-a36e5cd0bb23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:48.967141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdd77e-66bd-43cf-9cbc-a36e5cd0bb23 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:48.967418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdd77e-66bd-43cf-9cbc-a36e5cd0bb23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:48.967623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae815b4-b1b3-4b14-ab09-98c5a0eca459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd84504623f4ac6aa8ef7aa9156d6e2, user_id=f7bdba840d204dac8ae03756fbfdea10, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:48.967728 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdd77e-66bd-43cf-9cbc-a36e5cd0bb23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:48.970976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae815b4-b1b3-4b14-ab09-98c5a0eca459 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:48.981889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1139/2252] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:48 2022] PUT /identity/v3/projects/8cd84504623f4ac6aa8ef7aa9156d6e2/users/f7bdba840d204dac8ae03756fbfdea10/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:50:48.993261 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee7afa-1f83-48b1-8820-8ea319761647 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:48.993723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee7afa-1f83-48b1-8820-8ea319761647 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:48.994100 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee7afa-1f83-48b1-8820-8ea319761647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:49.025795 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-85ee7afa-1f83-48b1-8820-8ea319761647 None None] MFA Rules not processed for user `f7bdba840d204dac8ae03756fbfdea10`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:49.027229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-85ee7afa-1f83-48b1-8820-8ea319761647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:49.060796 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdd77e-66bd-43cf-9cbc-a36e5cd0bb23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:49.063634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdd77e-66bd-43cf-9cbc-a36e5cd0bb23 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:49.075735 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1114/2253] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:48 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:49.128779 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1140/2254] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:49.136957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdd77e-66bd-43cf-9cbc-a36e5cd0bb23 service cinder] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:49.147308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdd77e-66bd-43cf-9cbc-a36e5cd0bb23 service cinder] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:49.219189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7e7e4-9ea6-4b5d-8a11-b073c53e3a52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:49.220016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7e7e4-9ea6-4b5d-8a11-b073c53e3a52 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:49.220346 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7e7e4-9ea6-4b5d-8a11-b073c53e3a52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:49.220829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7e7e4-9ea6-4b5d-8a11-b073c53e3a52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:49.240210 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7e7e4-9ea6-4b5d-8a11-b073c53e3a52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:49.242686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7e7e4-9ea6-4b5d-8a11-b073c53e3a52 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:49.262111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1115/2255] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:50:49.394381 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee7afa-1f83-48b1-8820-8ea319761647 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:49.413144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee7afa-1f83-48b1-8820-8ea319761647 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:49.417999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fd240-1e9a-4c1d-b88f-900f23f39ca0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:49.418631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fd240-1e9a-4c1d-b88f-900f23f39ca0 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:49.418912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fd240-1e9a-4c1d-b88f-900f23f39ca0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:49.419188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fd240-1e9a-4c1d-b88f-900f23f39ca0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:49.421016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3c1fd240-1e9a-4c1d-b88f-900f23f39ca0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:49.502502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fd240-1e9a-4c1d-b88f-900f23f39ca0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:49.504230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fd240-1e9a-4c1d-b88f-900f23f39ca0 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:49.517973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1141/2256] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:51.611800 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6f3df-5e34-41a6-9b4b-4bc4d3ed51b2 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:51.612298 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6f3df-5e34-41a6-9b4b-4bc4d3ed51b2 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:51.612470 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6f3df-5e34-41a6-9b4b-4bc4d3ed51b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:51.633683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-c8d6f3df-5e34-41a6-9b4b-4bc4d3ed51b2 None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:51.634829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c8d6f3df-5e34-41a6-9b4b-4bc4d3ed51b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:51.653827 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:51.690839 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1116/2257] 10.222.0.70 () {58 vars in 967 bytes} [Wed Aug 24 16:50:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:51.703687 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fd240-1e9a-4c1d-b88f-900f23f39ca0 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:51.705121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3c1fd240-1e9a-4c1d-b88f-900f23f39ca0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:51.758818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:51.778280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fd240-1e9a-4c1d-b88f-900f23f39ca0 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:51.784704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7544f-a4a2-4fe3-809e-2523e4f61b64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:51.785861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7544f-a4a2-4fe3-809e-2523e4f61b64 service swift] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:51.786159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7544f-a4a2-4fe3-809e-2523e4f61b64 service swift] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:51.786483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7544f-a4a2-4fe3-809e-2523e4f61b64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:51.795617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7544f-a4a2-4fe3-809e-2523e4f61b64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:51.797275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7544f-a4a2-4fe3-809e-2523e4f61b64 service swift] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:51.805258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1142/2258] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 16:50:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:50:58.895178 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6f3df-5e34-41a6-9b4b-4bc4d3ed51b2 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:58.941142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44e1e3-e975-4129-8b2f-8c031e3162b0 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:58.941664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44e1e3-e975-4129-8b2f-8c031e3162b0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:58.941852 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44e1e3-e975-4129-8b2f-8c031e3162b0 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:58.943652 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1143/2259] 10.222.0.70 () {54 vars in 921 bytes} [Wed Aug 24 16:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:50:58.948273 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf267f-64f9-4e7b-9479-ff6115100317 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:58.948651 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf267f-64f9-4e7b-9479-ff6115100317 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:58.949134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf267f-64f9-4e7b-9479-ff6115100317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:58.962586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6f3df-5e34-41a6-9b4b-4bc4d3ed51b2 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:58.967816 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617522e-2ece-4488-8f86-e90b02e93970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:58.968585 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617522e-2ece-4488-8f86-e90b02e93970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:58.969432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617522e-2ece-4488-8f86-e90b02e93970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:58.969757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617522e-2ece-4488-8f86-e90b02e93970 admin admin] PATH_INFO: `/v3/users/507b3811edb540ea8c054b13bbe4ef05` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:58.972116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617522e-2ece-4488-8f86-e90b02e93970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=507b3811edb540ea8c054b13bbe4ef05)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:58.972252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-2caf267f-64f9-4e7b-9479-ff6115100317 None None] MFA Rules not processed for user `fc4c43c85a5d49979ba803fdb1041c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:58.972650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2caf267f-64f9-4e7b-9479-ff6115100317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:58.973136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617522e-2ece-4488-8f86-e90b02e93970 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:59.038892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1144/2260] 10.222.0.70 () {58 vars in 1018 bytes} [Wed Aug 24 16:50:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:59.050410 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf267f-64f9-4e7b-9479-ff6115100317 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:59.052871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2caf267f-64f9-4e7b-9479-ff6115100317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:59.067715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf267f-64f9-4e7b-9479-ff6115100317 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:59.074287 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262a348-7867-4fa6-95f5-ff2369ffb930 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:59.075137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262a348-7867-4fa6-95f5-ff2369ffb930 service placement] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:59.075447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262a348-7867-4fa6-95f5-ff2369ffb930 service placement] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:59.076554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262a348-7867-4fa6-95f5-ff2369ffb930 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:59.089365 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262a348-7867-4fa6-95f5-ff2369ffb930 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:59.090991 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262a348-7867-4fa6-95f5-ff2369ffb930 service placement] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:59.101484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1145/2261] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:50:59.110460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262a348-7867-4fa6-95f5-ff2369ffb930 service placement] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:59.110460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2262a348-7867-4fa6-95f5-ff2369ffb930 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:59.124312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262a348-7867-4fa6-95f5-ff2369ffb930 service placement] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:59.131586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd81a2-0d12-4e1e-ba4c-bcd75e18c7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:59.132571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd81a2-0d12-4e1e-ba4c-bcd75e18c7a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:59.132571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd81a2-0d12-4e1e-ba4c-bcd75e18c7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:59.132848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd81a2-0d12-4e1e-ba4c-bcd75e18c7a1 admin admin] PATH_INFO: `/v3/users/55975de527254dca90fc4f30bc50e393` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:59.134042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd81a2-0d12-4e1e-ba4c-bcd75e18c7a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55975de527254dca90fc4f30bc50e393)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:59.136514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd81a2-0d12-4e1e-ba4c-bcd75e18c7a1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:59.178021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c617522e-2ece-4488-8f86-e90b02e93970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '507b3811edb540ea8c054b13bbe4ef05'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:59.178734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c617522e-2ece-4488-8f86-e90b02e93970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '507b3811edb540ea8c054b13bbe4ef05'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:59.179341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c617522e-2ece-4488-8f86-e90b02e93970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '507b3811edb540ea8c054b13bbe4ef05'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:59.186438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c617522e-2ece-4488-8f86-e90b02e93970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '507b3811edb540ea8c054b13bbe4ef05'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:59.271946 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c617522e-2ece-4488-8f86-e90b02e93970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '507b3811edb540ea8c054b13bbe4ef05'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:59.290831 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1117/2262] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:58 2022] DELETE /identity/v3/users/507b3811edb540ea8c054b13bbe4ef05 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:59.295138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3168c-f040-4a10-acac-3e70261bebcf None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:59.295427 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3168c-f040-4a10-acac-3e70261bebcf None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:59.295703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3168c-f040-4a10-acac-3e70261bebcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:59.313861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-59f3168c-f040-4a10-acac-3e70261bebcf None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:59.315089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-59f3168c-f040-4a10-acac-3e70261bebcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:59.361690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:59.394446 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1118/2263] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:59.398945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d8cd81a2-0d12-4e1e-ba4c-bcd75e18c7a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55975de527254dca90fc4f30bc50e393'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:59.399953 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d8cd81a2-0d12-4e1e-ba4c-bcd75e18c7a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55975de527254dca90fc4f30bc50e393'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:59.400857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d8cd81a2-0d12-4e1e-ba4c-bcd75e18c7a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55975de527254dca90fc4f30bc50e393'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:59.404703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3168c-f040-4a10-acac-3e70261bebcf None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:59.408001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-59f3168c-f040-4a10-acac-3e70261bebcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:59.473387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3168c-f040-4a10-acac-3e70261bebcf None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:59.477282 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2090973-e69e-4630-a9fa-3dcada0496b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:59.477993 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2090973-e69e-4630-a9fa-3dcada0496b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:59.478372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2090973-e69e-4630-a9fa-3dcada0496b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:59.478761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2090973-e69e-4630-a9fa-3dcada0496b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:59.480616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c2090973-e69e-4630-a9fa-3dcada0496b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:59.493279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2090973-e69e-4630-a9fa-3dcada0496b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:59.494855 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2090973-e69e-4630-a9fa-3dcada0496b4 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:59.505126 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1119/2264] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:50:59.524137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d8cd81a2-0d12-4e1e-ba4c-bcd75e18c7a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55975de527254dca90fc4f30bc50e393'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:59.535207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d8cd81a2-0d12-4e1e-ba4c-bcd75e18c7a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55975de527254dca90fc4f30bc50e393'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:59.576788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1146/2265] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:59 2022] DELETE /identity/v3/users/55975de527254dca90fc4f30bc50e393 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:59.899463 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2090973-e69e-4630-a9fa-3dcada0496b4 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:59.899463 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c2090973-e69e-4630-a9fa-3dcada0496b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:59.911968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2090973-e69e-4630-a9fa-3dcada0496b4 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:59.979646 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb68450-817e-4e07-b4ea-059cde508f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:59.980247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb68450-817e-4e07-b4ea-059cde508f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:59.980569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb68450-817e-4e07-b4ea-059cde508f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:59.980846 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb68450-817e-4e07-b4ea-059cde508f7b admin admin] PATH_INFO: `/v3/projects/cd5a0fa1909747368101df31693c44e1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:59.981988 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb68450-817e-4e07-b4ea-059cde508f7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd5a0fa1909747368101df31693c44e1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:59.984007 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb68450-817e-4e07-b4ea-059cde508f7b admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:00.030803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd81a2-0d12-4e1e-ba4c-bcd75e18c7a1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:00.032369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d8cd81a2-0d12-4e1e-ba4c-bcd75e18c7a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:00.046660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd81a2-0d12-4e1e-ba4c-bcd75e18c7a1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:00.051904 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:00.052583 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:00.052977 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:00.053189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794 admin admin] PATH_INFO: `/v3/projects/2ada5631ac34434d91f15c0bc5489c44` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:00.054332 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ada5631ac34434d91f15c0bc5489c44)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:00.056318 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:00.103470 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ecb68450-817e-4e07-b4ea-059cde508f7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd5a0fa1909747368101df31693c44e1', 'request_id': 'req-ecb68450-817e-4e07-b4ea-059cde508f7b'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.110906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ecb68450-817e-4e07-b4ea-059cde508f7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd5a0fa1909747368101df31693c44e1', 'request_id': 'req-ecb68450-817e-4e07-b4ea-059cde508f7b'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.117316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ecb68450-817e-4e07-b4ea-059cde508f7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd5a0fa1909747368101df31693c44e1', 'request_id': 'req-ecb68450-817e-4e07-b4ea-059cde508f7b'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.152462 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:00.165086 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ada5631ac34434d91f15c0bc5489c44', 'request_id': 'req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.171643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ada5631ac34434d91f15c0bc5489c44', 'request_id': 'req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.177655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ada5631ac34434d91f15c0bc5489c44', 'request_id': 'req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.223863 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ecb68450-817e-4e07-b4ea-059cde508f7b admin admin] The token cache is being invalidate because project cd5a0fa1909747368101df31693c44e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:51:00.224156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ecb68450-817e-4e07-b4ea-059cde508f7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.226440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ecb68450-817e-4e07-b4ea-059cde508f7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.226440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1120/2266] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:59 2022] DELETE /identity/v3/projects/cd5a0fa1909747368101df31693c44e1 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:00.231410 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edabb1-ad38-4aa5-a4cd-ab792da69c37 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:00.231712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edabb1-ad38-4aa5-a4cd-ab792da69c37 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:00.231990 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edabb1-ad38-4aa5-a4cd-ab792da69c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:00.252287 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-86edabb1-ad38-4aa5-a4cd-ab792da69c37 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:00.253504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-86edabb1-ad38-4aa5-a4cd-ab792da69c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:00.270047 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:00.305072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794 admin admin] The token cache is being invalidate because project 2ada5631ac34434d91f15c0bc5489c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:51:00.305370 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.306066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.308234 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1147/2267] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:51:00 2022] DELETE /identity/v3/projects/2ada5631ac34434d91f15c0bc5489c44 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:51:00.312018 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:00.314088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:00.314960 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1121/2268] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:51:00.322726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f4db2-232d-4661-9126-d5bd8af5dbe5 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:00.323072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f4db2-232d-4661-9126-d5bd8af5dbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:00.323352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f4db2-232d-4661-9126-d5bd8af5dbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:00.327581 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d05c17-23bb-4dfa-9055-a1e3f08c7794 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:00.333243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84f190-96dc-41d8-ada3-8121721595f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:00.334077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84f190-96dc-41d8-ada3-8121721595f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:00.334361 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84f190-96dc-41d8-ada3-8121721595f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:00.334637 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84f190-96dc-41d8-ada3-8121721595f9 admin admin] PATH_INFO: `/v3/users/edd9277d9d744222b9b8fbc74e97dd41` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:00.335773 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84f190-96dc-41d8-ada3-8121721595f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edd9277d9d744222b9b8fbc74e97dd41)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:00.337782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84f190-96dc-41d8-ada3-8121721595f9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:00.342507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-4b7f4db2-232d-4661-9126-d5bd8af5dbe5 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:00.343706 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-4b7f4db2-232d-4661-9126-d5bd8af5dbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:00.352115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1122/2269] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:00.356390 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f4db2-232d-4661-9126-d5bd8af5dbe5 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:00.366127 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f4db2-232d-4661-9126-d5bd8af5dbe5 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:00.370761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015dacd-7761-4953-b010-a64703b1394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:00.371375 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015dacd-7761-4953-b010-a64703b1394f admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:00.375995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015dacd-7761-4953-b010-a64703b1394f admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:00.375995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015dacd-7761-4953-b010-a64703b1394f admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:00.375995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015dacd-7761-4953-b010-a64703b1394f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:51:00.375995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015dacd-7761-4953-b010-a64703b1394f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:00.375995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015dacd-7761-4953-b010-a64703b1394f admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:00.386319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1123/2270] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:51:00.390472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015dacd-7761-4953-b010-a64703b1394f admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:00.399689 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015dacd-7761-4953-b010-a64703b1394f admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:00.404475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9576f-a4f9-40fb-840c-d2f85706ff37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:00.404981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9576f-a4f9-40fb-840c-d2f85706ff37 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:00.405263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9576f-a4f9-40fb-840c-d2f85706ff37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:00.405537 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9576f-a4f9-40fb-840c-d2f85706ff37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:00.406137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9576f-a4f9-40fb-840c-d2f85706ff37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:00.408524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9576f-a4f9-40fb-840c-d2f85706ff37 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:00.499447 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1124/2271] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:51:00 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:51:00.500385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9576f-a4f9-40fb-840c-d2f85706ff37 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:00.501970 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-07e9576f-a4f9-40fb-840c-d2f85706ff37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:00.528792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ae84f190-96dc-41d8-ada3-8121721595f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edd9277d9d744222b9b8fbc74e97dd41'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.529491 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ae84f190-96dc-41d8-ada3-8121721595f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edd9277d9d744222b9b8fbc74e97dd41'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.530260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ae84f190-96dc-41d8-ada3-8121721595f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edd9277d9d744222b9b8fbc74e97dd41'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.563806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9576f-a4f9-40fb-840c-d2f85706ff37 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:00.568429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3aea33-ea6c-4c8f-8059-869927792acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:00.569067 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3aea33-ea6c-4c8f-8059-869927792acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:00.569342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3aea33-ea6c-4c8f-8059-869927792acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:00.569610 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3aea33-ea6c-4c8f-8059-869927792acc admin admin] PATH_INFO: `/v3/users/0b5fec844d8f4985aebb5c314f9d3cb5` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:00.570782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3aea33-ea6c-4c8f-8059-869927792acc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b5fec844d8f4985aebb5c314f9d3cb5)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:00.572719 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3aea33-ea6c-4c8f-8059-869927792acc admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:00.589091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ae84f190-96dc-41d8-ada3-8121721595f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edd9277d9d744222b9b8fbc74e97dd41'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.596899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ae84f190-96dc-41d8-ada3-8121721595f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edd9277d9d744222b9b8fbc74e97dd41'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.614259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1148/2272] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:51:00 2022] DELETE /identity/v3/users/edd9277d9d744222b9b8fbc74e97dd41 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:51:00.615089 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84f190-96dc-41d8-ada3-8121721595f9 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:00.616310 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ae84f190-96dc-41d8-ada3-8121721595f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:00.680571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84f190-96dc-41d8-ada3-8121721595f9 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:00.685732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f0a3e-9e6a-4921-a5df-5d246347e248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:00.686425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f0a3e-9e6a-4921-a5df-5d246347e248 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:00.686821 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f0a3e-9e6a-4921-a5df-5d246347e248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:00.687164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f0a3e-9e6a-4921-a5df-5d246347e248 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:00.687880 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f0a3e-9e6a-4921-a5df-5d246347e248 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:00.689883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f0a3e-9e6a-4921-a5df-5d246347e248 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:00.772149 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad3aea33-ea6c-4c8f-8059-869927792acc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b5fec844d8f4985aebb5c314f9d3cb5'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.773028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad3aea33-ea6c-4c8f-8059-869927792acc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b5fec844d8f4985aebb5c314f9d3cb5'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.773634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad3aea33-ea6c-4c8f-8059-869927792acc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b5fec844d8f4985aebb5c314f9d3cb5'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.780812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad3aea33-ea6c-4c8f-8059-869927792acc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b5fec844d8f4985aebb5c314f9d3cb5'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.799525 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1149/2273] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:51:00 2022] POST /identity/v3/users => generated 286 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:00.803958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f0a3e-9e6a-4921-a5df-5d246347e248 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:00.807740 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3a0f0a3e-9e6a-4921-a5df-5d246347e248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:00.820198 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f0a3e-9e6a-4921-a5df-5d246347e248 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:00.825021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535da72-f8e6-4633-b7e6-901280d3e36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:00.825638 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535da72-f8e6-4633-b7e6-901280d3e36f admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:00.830624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535da72-f8e6-4633-b7e6-901280d3e36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:00.830624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535da72-f8e6-4633-b7e6-901280d3e36f admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:00.830624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535da72-f8e6-4633-b7e6-901280d3e36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:00.830624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535da72-f8e6-4633-b7e6-901280d3e36f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:00.842777 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1150/2274] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:00.847658 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535da72-f8e6-4633-b7e6-901280d3e36f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:00.849098 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad3aea33-ea6c-4c8f-8059-869927792acc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b5fec844d8f4985aebb5c314f9d3cb5'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:00.859354 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535da72-f8e6-4633-b7e6-901280d3e36f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:00.864273 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fb1dd-c267-40d4-9114-83ffd223ec9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:00.864934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fb1dd-c267-40d4-9114-83ffd223ec9a admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:00.865213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fb1dd-c267-40d4-9114-83ffd223ec9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:00.865488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fb1dd-c267-40d4-9114-83ffd223ec9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:00.866042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:00.866167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4fb1dd-c267-40d4-9114-83ffd223ec9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:00.867880 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4fb1dd-c267-40d4-9114-83ffd223ec9a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:00.870474 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=80,keystone:DELETE=15,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:00.871145 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1125/2275] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:51:00 2022] DELETE /identity/v3/users/0b5fec844d8f4985aebb5c314f9d3cb5 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:00.876424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5946b9-e76f-4f42-a673-2d2991fd0313 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:00.876716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5946b9-e76f-4f42-a673-2d2991fd0313 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:00.877166 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5946b9-e76f-4f42-a673-2d2991fd0313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:00.880446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1151/2276] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:00.884627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fb1dd-c267-40d4-9114-83ffd223ec9a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:00.896912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fb1dd-c267-40d4-9114-83ffd223ec9a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:00.902957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-3d5946b9-e76f-4f42-a673-2d2991fd0313 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:00.903710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3d5946b9-e76f-4f42-a673-2d2991fd0313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:00.967255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce34cf-d366-41bd-91fd-24e3547c9e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:00.967832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce34cf-d366-41bd-91fd-24e3547c9e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:00.968115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce34cf-d366-41bd-91fd-24e3547c9e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:00.968470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce34cf-d366-41bd-91fd-24e3547c9e6c admin admin] PATH_INFO: `/v3/projects/d08617719a5947ddbbf208f4dcdde5e6/users/78486adda1424457b0c6d5d91e69403e/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:00.970967 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1126/2277] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:00.980549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5946b9-e76f-4f42-a673-2d2991fd0313 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:00.981882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3d5946b9-e76f-4f42-a673-2d2991fd0313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:00.984414 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce34cf-d366-41bd-91fd-24e3547c9e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d08617719a5947ddbbf208f4dcdde5e6, user_id=78486adda1424457b0c6d5d91e69403e, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:00.987243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce34cf-d366-41bd-91fd-24e3547c9e6c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:01.036804 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5946b9-e76f-4f42-a673-2d2991fd0313 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:01.040594 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5de363-3c56-4558-a317-753fc551fef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:01.041204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5de363-3c56-4558-a317-753fc551fef7 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:01.041484 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5de363-3c56-4558-a317-753fc551fef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:01.041759 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5de363-3c56-4558-a317-753fc551fef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:01.050720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5de363-3c56-4558-a317-753fc551fef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:01.052393 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5de363-3c56-4558-a317-753fc551fef7 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:01.061672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1127/2278] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:51:01.082706 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5de363-3c56-4558-a317-753fc551fef7 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:01.082706 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6f5de363-3c56-4558-a317-753fc551fef7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:01.092560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1152/2279] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:51:00 2022] PUT /identity/v3/projects/d08617719a5947ddbbf208f4dcdde5e6/users/78486adda1424457b0c6d5d91e69403e/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:01.103670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5de363-3c56-4558-a317-753fc551fef7 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:01.115360 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76296bd9-c70e-4d90-af12-89e074359601 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:01.116333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76296bd9-c70e-4d90-af12-89e074359601 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:01.116775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76296bd9-c70e-4d90-af12-89e074359601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:01.152168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-76296bd9-c70e-4d90-af12-89e074359601 None None] MFA Rules not processed for user `78486adda1424457b0c6d5d91e69403e`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:01.153425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-76296bd9-c70e-4d90-af12-89e074359601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:01.174177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a6224-3174-458c-94da-44bbe7e586b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:01.174816 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a6224-3174-458c-94da-44bbe7e586b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:01.175099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a6224-3174-458c-94da-44bbe7e586b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:01.175377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a6224-3174-458c-94da-44bbe7e586b6 admin admin] PATH_INFO: `/v3/projects/b2d9509e669a41cca873297a8131b94d` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:01.176810 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a6224-3174-458c-94da-44bbe7e586b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2d9509e669a41cca873297a8131b94d)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:01.180236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a6224-3174-458c-94da-44bbe7e586b6 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:01.230398 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1153/2280] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:01.242095 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76296bd9-c70e-4d90-af12-89e074359601 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:01.243481 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-76296bd9-c70e-4d90-af12-89e074359601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:01.300723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76296bd9-c70e-4d90-af12-89e074359601 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:01.304334 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a78d54-895c-4da5-8403-be7ec0990113 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56418ff737504181a59714aad9f1c375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:01.305171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a78d54-895c-4da5-8403-be7ec0990113 service glance] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:01.305302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a78d54-895c-4da5-8403-be7ec0990113 service glance] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:01.305545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a78d54-895c-4da5-8403-be7ec0990113 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:01.315076 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a78d54-895c-4da5-8403-be7ec0990113 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:01.316609 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a78d54-895c-4da5-8403-be7ec0990113 service glance] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:01.327384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1154/2281] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:51:01.336660 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad3a6224-3174-458c-94da-44bbe7e586b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2d9509e669a41cca873297a8131b94d', 'request_id': 'req-ad3a6224-3174-458c-94da-44bbe7e586b6'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:01.343548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad3a6224-3174-458c-94da-44bbe7e586b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2d9509e669a41cca873297a8131b94d', 'request_id': 'req-ad3a6224-3174-458c-94da-44bbe7e586b6'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:01.349808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad3a6224-3174-458c-94da-44bbe7e586b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2d9509e669a41cca873297a8131b94d', 'request_id': 'req-ad3a6224-3174-458c-94da-44bbe7e586b6'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:01.369565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:01.447269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad3a6224-3174-458c-94da-44bbe7e586b6 admin admin] The token cache is being invalidate because project b2d9509e669a41cca873297a8131b94d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:51:01.447269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad3a6224-3174-458c-94da-44bbe7e586b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:01.447269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ad3a6224-3174-458c-94da-44bbe7e586b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:01.447269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1128/2282] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:51:01 2022] DELETE /identity/v3/projects/b2d9509e669a41cca873297a8131b94d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:01.447553 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a78d54-895c-4da5-8403-be7ec0990113 service glance] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:01.457628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a78d54-895c-4da5-8403-be7ec0990113 service glance] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:01.494851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:01.518273 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb3644-4e6a-46a3-a6fe-30918576887c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:01.518969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb3644-4e6a-46a3-a6fe-30918576887c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:01.519320 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb3644-4e6a-46a3-a6fe-30918576887c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:01.519661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb3644-4e6a-46a3-a6fe-30918576887c admin admin] PATH_INFO: `/v3/users/f7bdba840d204dac8ae03756fbfdea10` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:01.520893 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb3644-4e6a-46a3-a6fe-30918576887c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7bdba840d204dac8ae03756fbfdea10)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:01.522933 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb3644-4e6a-46a3-a6fe-30918576887c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:01.531197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a6224-3174-458c-94da-44bbe7e586b6 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:01.533728 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ad3a6224-3174-458c-94da-44bbe7e586b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:01.557150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a6224-3174-458c-94da-44bbe7e586b6 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:01.564187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ebc44-29cb-4797-be2f-0244430d2353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:01.565215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ebc44-29cb-4797-be2f-0244430d2353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:01.565215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ebc44-29cb-4797-be2f-0244430d2353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:01.565412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ebc44-29cb-4797-be2f-0244430d2353 admin admin] PATH_INFO: `/v3/projects/57333e5df7104c449effa0d6c975b78d` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:01.567143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ebc44-29cb-4797-be2f-0244430d2353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57333e5df7104c449effa0d6c975b78d)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:01.569304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ebc44-29cb-4797-be2f-0244430d2353 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:01.751928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a29ebc44-29cb-4797-be2f-0244430d2353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57333e5df7104c449effa0d6c975b78d', 'request_id': 'req-a29ebc44-29cb-4797-be2f-0244430d2353'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:01.754185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9fdb3644-4e6a-46a3-a6fe-30918576887c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7bdba840d204dac8ae03756fbfdea10'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:01.754923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9fdb3644-4e6a-46a3-a6fe-30918576887c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7bdba840d204dac8ae03756fbfdea10'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:01.755597 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9fdb3644-4e6a-46a3-a6fe-30918576887c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7bdba840d204dac8ae03756fbfdea10'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:01.758333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a29ebc44-29cb-4797-be2f-0244430d2353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57333e5df7104c449effa0d6c975b78d', 'request_id': 'req-a29ebc44-29cb-4797-be2f-0244430d2353'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:01.763865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a29ebc44-29cb-4797-be2f-0244430d2353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57333e5df7104c449effa0d6c975b78d', 'request_id': 'req-a29ebc44-29cb-4797-be2f-0244430d2353'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:01.877660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9fdb3644-4e6a-46a3-a6fe-30918576887c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7bdba840d204dac8ae03756fbfdea10'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:01.885544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9fdb3644-4e6a-46a3-a6fe-30918576887c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7bdba840d204dac8ae03756fbfdea10'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:01.905798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1155/2283] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:51:01 2022] DELETE /identity/v3/users/f7bdba840d204dac8ae03756fbfdea10 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:01.967062 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a29ebc44-29cb-4797-be2f-0244430d2353 admin admin] The token cache is being invalidate because project 57333e5df7104c449effa0d6c975b78d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:51:01.967620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a29ebc44-29cb-4797-be2f-0244430d2353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:01.968105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a29ebc44-29cb-4797-be2f-0244430d2353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:01.970497 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1129/2284] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:51:01 2022] DELETE /identity/v3/projects/57333e5df7104c449effa0d6c975b78d => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:02.447155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb3644-4e6a-46a3-a6fe-30918576887c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:02.448755 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9fdb3644-4e6a-46a3-a6fe-30918576887c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:02.517260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb3644-4e6a-46a3-a6fe-30918576887c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:02.521799 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485c47e-408a-4ae8-bc14-da864538de0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:02.522431 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485c47e-408a-4ae8-bc14-da864538de0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:02.522713 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485c47e-408a-4ae8-bc14-da864538de0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:02.523055 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485c47e-408a-4ae8-bc14-da864538de0b admin admin] PATH_INFO: `/v3/projects/8cd84504623f4ac6aa8ef7aa9156d6e2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:02.524174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485c47e-408a-4ae8-bc14-da864538de0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cd84504623f4ac6aa8ef7aa9156d6e2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:02.526130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485c47e-408a-4ae8-bc14-da864538de0b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:02.638300 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:02.648584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d485c47e-408a-4ae8-bc14-da864538de0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cd84504623f4ac6aa8ef7aa9156d6e2', 'request_id': 'req-d485c47e-408a-4ae8-bc14-da864538de0b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:02.655119 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d485c47e-408a-4ae8-bc14-da864538de0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cd84504623f4ac6aa8ef7aa9156d6e2', 'request_id': 'req-d485c47e-408a-4ae8-bc14-da864538de0b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:02.660889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d485c47e-408a-4ae8-bc14-da864538de0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cd84504623f4ac6aa8ef7aa9156d6e2', 'request_id': 'req-d485c47e-408a-4ae8-bc14-da864538de0b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:02.733320 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d485c47e-408a-4ae8-bc14-da864538de0b admin admin] The token cache is being invalidate because project 8cd84504623f4ac6aa8ef7aa9156d6e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:51:02.733643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d485c47e-408a-4ae8-bc14-da864538de0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:02.734295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d485c47e-408a-4ae8-bc14-da864538de0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:02.739369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1156/2285] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:51:02 2022] DELETE /identity/v3/projects/8cd84504623f4ac6aa8ef7aa9156d6e2 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:02.742221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e95daa-1186-4c1e-abe8-072f105bddf0 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:02.742684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e95daa-1186-4c1e-abe8-072f105bddf0 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:02.742962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e95daa-1186-4c1e-abe8-072f105bddf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:02.763993 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-b5e95daa-1186-4c1e-abe8-072f105bddf0 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:02.765232 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b5e95daa-1186-4c1e-abe8-072f105bddf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:02.836735 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1130/2286] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:02.845134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484d178-dc68-4dba-86b1-85a33bdb3ae1 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:02.845519 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484d178-dc68-4dba-86b1-85a33bdb3ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:02.845842 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484d178-dc68-4dba-86b1-85a33bdb3ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:02.868664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-2484d178-dc68-4dba-86b1-85a33bdb3ae1 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:02.869914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2484d178-dc68-4dba-86b1-85a33bdb3ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:02.879229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1157/2287] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:02.883500 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e95daa-1186-4c1e-abe8-072f105bddf0 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:02.894218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e95daa-1186-4c1e-abe8-072f105bddf0 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:02.899032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b769ed-b5d4-49fc-a7b0-ffea8f7876d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:02.899640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b769ed-b5d4-49fc-a7b0-ffea8f7876d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:02.899924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b769ed-b5d4-49fc-a7b0-ffea8f7876d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:02.900201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b769ed-b5d4-49fc-a7b0-ffea8f7876d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:02.900888 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b769ed-b5d4-49fc-a7b0-ffea8f7876d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:51:02.901196 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b769ed-b5d4-49fc-a7b0-ffea8f7876d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:02.902924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b769ed-b5d4-49fc-a7b0-ffea8f7876d2 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:02.923371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1131/2288] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:51:02.924697 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484d178-dc68-4dba-86b1-85a33bdb3ae1 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:02.934363 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484d178-dc68-4dba-86b1-85a33bdb3ae1 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:02.940493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb20f8d-2e9d-4a43-b873-6ee3583871f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:02.941103 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb20f8d-2e9d-4a43-b873-6ee3583871f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:02.941386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb20f8d-2e9d-4a43-b873-6ee3583871f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:02.941821 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb20f8d-2e9d-4a43-b873-6ee3583871f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:02.945949 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb20f8d-2e9d-4a43-b873-6ee3583871f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:02.945949 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb20f8d-2e9d-4a43-b873-6ee3583871f9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:03.053732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1158/2289] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:51:02 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:03.055202 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b769ed-b5d4-49fc-a7b0-ffea8f7876d2 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:03.065632 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b769ed-b5d4-49fc-a7b0-ffea8f7876d2 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:03.123860 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:03.140577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b91c50-c12b-44f1-bd9c-b1ec0d3195ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:03.141894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b91c50-c12b-44f1-bd9c-b1ec0d3195ea admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:03.142046 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b91c50-c12b-44f1-bd9c-b1ec0d3195ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:03.152006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b91c50-c12b-44f1-bd9c-b1ec0d3195ea admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:03.152761 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b91c50-c12b-44f1-bd9c-b1ec0d3195ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:03.154770 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b91c50-c12b-44f1-bd9c-b1ec0d3195ea admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:03.280900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1132/2290] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:51:03 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:03.286100 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb20f8d-2e9d-4a43-b873-6ee3583871f9 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:03.301836 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb20f8d-2e9d-4a43-b873-6ee3583871f9 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:03.322992 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2eda77-3c2b-4b31-8759-98139ad866ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:03.323650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eda77-3c2b-4b31-8759-98139ad866ea admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:03.323908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eda77-3c2b-4b31-8759-98139ad866ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:03.324182 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eda77-3c2b-4b31-8759-98139ad866ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:03.324876 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2eda77-3c2b-4b31-8759-98139ad866ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:03.326522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2eda77-3c2b-4b31-8759-98139ad866ea admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:03.339281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1159/2291] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:51:03.344803 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b91c50-c12b-44f1-bd9c-b1ec0d3195ea admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:03.355528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b91c50-c12b-44f1-bd9c-b1ec0d3195ea admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:03.361586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb08aa-bab9-4286-9061-c9e199beb8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:03.365094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb08aa-bab9-4286-9061-c9e199beb8dc admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:03.365094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb08aa-bab9-4286-9061-c9e199beb8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:03.365094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb08aa-bab9-4286-9061-c9e199beb8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:03.365094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb08aa-bab9-4286-9061-c9e199beb8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:03.365448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb08aa-bab9-4286-9061-c9e199beb8dc admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:03.379121 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1133/2292] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:03.383336 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2eda77-3c2b-4b31-8759-98139ad866ea admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:03.394680 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2eda77-3c2b-4b31-8759-98139ad866ea admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:03.399447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea31728-41c6-4bb9-8ca0-f0c9133aff1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:03.400182 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea31728-41c6-4bb9-8ca0-f0c9133aff1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:03.400598 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea31728-41c6-4bb9-8ca0-f0c9133aff1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:03.400943 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea31728-41c6-4bb9-8ca0-f0c9133aff1d admin admin] PATH_INFO: `/v3/projects/4af66e7ae0594f608ecdbcaa4e1d3625/users/f0f26b3467994912a051e96f880c7783/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:03.415751 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea31728-41c6-4bb9-8ca0-f0c9133aff1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af66e7ae0594f608ecdbcaa4e1d3625, user_id=f0f26b3467994912a051e96f880c7783, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:03.418655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea31728-41c6-4bb9-8ca0-f0c9133aff1d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:03.496769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1160/2293] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:51:03 2022] PUT /identity/v3/projects/4af66e7ae0594f608ecdbcaa4e1d3625/users/f0f26b3467994912a051e96f880c7783/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:03.510770 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3d548-3e3c-477d-a106-11830c27b4e2 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:03.511198 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3d548-3e3c-477d-a106-11830c27b4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:03.511578 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3d548-3e3c-477d-a106-11830c27b4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:03.542305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-51f3d548-3e3c-477d-a106-11830c27b4e2 None None] MFA Rules not processed for user `f0f26b3467994912a051e96f880c7783`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:03.544700 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-51f3d548-3e3c-477d-a106-11830c27b4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:03.614225 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1134/2294] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:03.626025 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea31728-41c6-4bb9-8ca0-f0c9133aff1d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:03.627735 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bea31728-41c6-4bb9-8ca0-f0c9133aff1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:03.688664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea31728-41c6-4bb9-8ca0-f0c9133aff1d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:03.693437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c7a96-82cb-42ac-97eb-a4822c15f618 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:03.694114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c7a96-82cb-42ac-97eb-a4822c15f618 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:03.694512 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c7a96-82cb-42ac-97eb-a4822c15f618 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:03.694825 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c7a96-82cb-42ac-97eb-a4822c15f618 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:03.704353 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c7a96-82cb-42ac-97eb-a4822c15f618 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:03.705825 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c7a96-82cb-42ac-97eb-a4822c15f618 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:03.715230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1161/2295] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:04.983190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd66dc0-1b7a-4bed-8736-cc1be308bcd6 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:04.983670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd66dc0-1b7a-4bed-8736-cc1be308bcd6 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:04.983779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd66dc0-1b7a-4bed-8736-cc1be308bcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:05.003199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-fcd66dc0-1b7a-4bed-8736-cc1be308bcd6 None None] MFA Rules not processed for user `45913fc3d9ba42ea9e971db25e656c61`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:05.004426 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-fcd66dc0-1b7a-4bed-8736-cc1be308bcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:05.059388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1135/2296] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:07.380575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c7a96-82cb-42ac-97eb-a4822c15f618 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:07.382019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-de1c7a96-82cb-42ac-97eb-a4822c15f618 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:07.448340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:07.452756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c7a96-82cb-42ac-97eb-a4822c15f618 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:07.457788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfeb0c-7821-43da-bef3-1fef866b220a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:07.460442 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfeb0c-7821-43da-bef3-1fef866b220a service placement] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:07.460442 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfeb0c-7821-43da-bef3-1fef866b220a service placement] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:07.460442 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfeb0c-7821-43da-bef3-1fef866b220a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:07.460779 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a9dfeb0c-7821-43da-bef3-1fef866b220a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:07.472817 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfeb0c-7821-43da-bef3-1fef866b220a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:07.474384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfeb0c-7821-43da-bef3-1fef866b220a service placement] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:07.485191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1162/2297] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:15.052870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:17.525982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:20.107909 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd66dc0-1b7a-4bed-8736-cc1be308bcd6 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:20.171373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd66dc0-1b7a-4bed-8736-cc1be308bcd6 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:20.176050 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a1b0f-f3b5-4f09-94d6-da599297f0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:20.176720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a1b0f-f3b5-4f09-94d6-da599297f0f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:20.177002 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a1b0f-f3b5-4f09-94d6-da599297f0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:20.177275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a1b0f-f3b5-4f09-94d6-da599297f0f6 admin admin] PATH_INFO: `/v3/users/f0f26b3467994912a051e96f880c7783` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:20.178968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a1b0f-f3b5-4f09-94d6-da599297f0f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0f26b3467994912a051e96f880c7783)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:20.180963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a1b0f-f3b5-4f09-94d6-da599297f0f6 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:20.388678 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2d6a1b0f-f3b5-4f09-94d6-da599297f0f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0f26b3467994912a051e96f880c7783'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:20.389390 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2d6a1b0f-f3b5-4f09-94d6-da599297f0f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0f26b3467994912a051e96f880c7783'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:20.390005 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2d6a1b0f-f3b5-4f09-94d6-da599297f0f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0f26b3467994912a051e96f880c7783'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:20.397026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2d6a1b0f-f3b5-4f09-94d6-da599297f0f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0f26b3467994912a051e96f880c7783'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:20.483151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2d6a1b0f-f3b5-4f09-94d6-da599297f0f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0f26b3467994912a051e96f880c7783'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:20.501605 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1136/2298] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:51:20 2022] DELETE /identity/v3/users/f0f26b3467994912a051e96f880c7783 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:20.506185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062eaaf-ac87-44a5-80c4-2be778dd3b57 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:20.510590 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062eaaf-ac87-44a5-80c4-2be778dd3b57 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:20.510590 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062eaaf-ac87-44a5-80c4-2be778dd3b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:20.525856 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c062eaaf-ac87-44a5-80c4-2be778dd3b57 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:20.527248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c062eaaf-ac87-44a5-80c4-2be778dd3b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:20.586967 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1163/2299] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:20.596623 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a1b0f-f3b5-4f09-94d6-da599297f0f6 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:20.598016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2d6a1b0f-f3b5-4f09-94d6-da599297f0f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:20.655242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a1b0f-f3b5-4f09-94d6-da599297f0f6 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:20.658903 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360e9b3-fb4a-474b-b628-b83bcfb8b3aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:20.659506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360e9b3-fb4a-474b-b628-b83bcfb8b3aa service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:20.659782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360e9b3-fb4a-474b-b628-b83bcfb8b3aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:20.660057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360e9b3-fb4a-474b-b628-b83bcfb8b3aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:20.669442 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360e9b3-fb4a-474b-b628-b83bcfb8b3aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:20.670862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360e9b3-fb4a-474b-b628-b83bcfb8b3aa service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:20.680878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1137/2300] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:21.100412 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062eaaf-ac87-44a5-80c4-2be778dd3b57 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:21.101979 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c062eaaf-ac87-44a5-80c4-2be778dd3b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:21.115928 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062eaaf-ac87-44a5-80c4-2be778dd3b57 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:21.120628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1236871-eccf-49ad-a797-bf1479f9670e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:21.121274 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1236871-eccf-49ad-a797-bf1479f9670e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:21.121559 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1236871-eccf-49ad-a797-bf1479f9670e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:21.121841 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1236871-eccf-49ad-a797-bf1479f9670e admin admin] PATH_INFO: `/v3/projects/4af66e7ae0594f608ecdbcaa4e1d3625` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:21.123247 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1236871-eccf-49ad-a797-bf1479f9670e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4af66e7ae0594f608ecdbcaa4e1d3625)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:21.125395 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1236871-eccf-49ad-a797-bf1479f9670e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:21.226389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d1236871-eccf-49ad-a797-bf1479f9670e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4af66e7ae0594f608ecdbcaa4e1d3625', 'request_id': 'req-d1236871-eccf-49ad-a797-bf1479f9670e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:21.232788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d1236871-eccf-49ad-a797-bf1479f9670e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4af66e7ae0594f608ecdbcaa4e1d3625', 'request_id': 'req-d1236871-eccf-49ad-a797-bf1479f9670e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:21.245069 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d1236871-eccf-49ad-a797-bf1479f9670e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4af66e7ae0594f608ecdbcaa4e1d3625', 'request_id': 'req-d1236871-eccf-49ad-a797-bf1479f9670e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:21.322921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d1236871-eccf-49ad-a797-bf1479f9670e admin admin] The token cache is being invalidate because project 4af66e7ae0594f608ecdbcaa4e1d3625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:51:21.323217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d1236871-eccf-49ad-a797-bf1479f9670e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:21.323877 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d1236871-eccf-49ad-a797-bf1479f9670e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:21.325398 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1164/2301] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:51:21 2022] DELETE /identity/v3/projects/4af66e7ae0594f608ecdbcaa4e1d3625 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:21.330690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6716ff-bd97-442a-a850-5341ceb46d10 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:21.331155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6716ff-bd97-442a-a850-5341ceb46d10 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:21.331256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6716ff-bd97-442a-a850-5341ceb46d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:21.350963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-ae6716ff-bd97-442a-a850-5341ceb46d10 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:21.356124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ae6716ff-bd97-442a-a850-5341ceb46d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:21.368910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:21.411169 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1138/2302] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:21.419408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0c661-c410-4064-95bc-ae88d4616961 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:21.419698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0c661-c410-4064-95bc-ae88d4616961 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:21.420004 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0c661-c410-4064-95bc-ae88d4616961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:21.440554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4af0c661-c410-4064-95bc-ae88d4616961 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:21.441677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4af0c661-c410-4064-95bc-ae88d4616961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:21.449995 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1165/2303] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:21.454282 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6716ff-bd97-442a-a850-5341ceb46d10 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:21.464629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6716ff-bd97-442a-a850-5341ceb46d10 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:21.469786 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749f84e-1a4c-42de-9584-a330a1cf5315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:21.470799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749f84e-1a4c-42de-9584-a330a1cf5315 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:21.470799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749f84e-1a4c-42de-9584-a330a1cf5315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:21.470999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749f84e-1a4c-42de-9584-a330a1cf5315 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:21.471522 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749f84e-1a4c-42de-9584-a330a1cf5315 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:51:21.471826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749f84e-1a4c-42de-9584-a330a1cf5315 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:21.473464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749f84e-1a4c-42de-9584-a330a1cf5315 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:21.484828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1139/2304] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:51:21.489242 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0c661-c410-4064-95bc-ae88d4616961 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:21.498794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0c661-c410-4064-95bc-ae88d4616961 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:21.503438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a237416-41ef-4287-a861-1d57b37ee2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:21.504033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a237416-41ef-4287-a861-1d57b37ee2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:21.504317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a237416-41ef-4287-a861-1d57b37ee2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:21.504642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a237416-41ef-4287-a861-1d57b37ee2d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:21.505298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a237416-41ef-4287-a861-1d57b37ee2d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:21.507663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a237416-41ef-4287-a861-1d57b37ee2d9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:21.599352 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1166/2305] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:51:21 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:51:21.603328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749f84e-1a4c-42de-9584-a330a1cf5315 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:21.613329 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749f84e-1a4c-42de-9584-a330a1cf5315 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:21.668981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a6d56-73ff-4a29-824b-c2801aa524c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:21.669584 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a6d56-73ff-4a29-824b-c2801aa524c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:21.669868 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a6d56-73ff-4a29-824b-c2801aa524c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:21.670143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a6d56-73ff-4a29-824b-c2801aa524c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:21.670775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a6d56-73ff-4a29-824b-c2801aa524c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:21.672718 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a6d56-73ff-4a29-824b-c2801aa524c1 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:21.768787 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1140/2306] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:51:21 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:21.772957 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a237416-41ef-4287-a861-1d57b37ee2d9 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:21.783322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a237416-41ef-4287-a861-1d57b37ee2d9 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:21.787965 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd475e-5245-490b-8e80-1e85bd0ca0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:21.788596 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd475e-5245-490b-8e80-1e85bd0ca0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:21.788882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd475e-5245-490b-8e80-1e85bd0ca0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:21.789159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd475e-5245-490b-8e80-1e85bd0ca0a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:21.789840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd475e-5245-490b-8e80-1e85bd0ca0a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:21.791423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd475e-5245-490b-8e80-1e85bd0ca0a5 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:21.801675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1167/2307] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:21.806237 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a6d56-73ff-4a29-824b-c2801aa524c1 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:21.816165 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a6d56-73ff-4a29-824b-c2801aa524c1 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:21.821490 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661379dd-d18f-4e14-9974-e050b2f9b0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:21.822082 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661379dd-d18f-4e14-9974-e050b2f9b0b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:21.822365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661379dd-d18f-4e14-9974-e050b2f9b0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:21.822644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661379dd-d18f-4e14-9974-e050b2f9b0b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:21.823284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661379dd-d18f-4e14-9974-e050b2f9b0b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:21.824991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661379dd-d18f-4e14-9974-e050b2f9b0b8 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:21.835386 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1141/2308] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:21.839474 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd475e-5245-490b-8e80-1e85bd0ca0a5 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:21.849347 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd475e-5245-490b-8e80-1e85bd0ca0a5 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:21.853935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31b97f-9cbb-45c1-b49b-b1b4c977c445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:21.854577 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31b97f-9cbb-45c1-b49b-b1b4c977c445 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:21.854859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31b97f-9cbb-45c1-b49b-b1b4c977c445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:21.855134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31b97f-9cbb-45c1-b49b-b1b4c977c445 admin admin] PATH_INFO: `/v3/projects/1715333032634911bcf0f7a41fab587a/users/90e3f9b6511347a692bed976865d5fad/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:21.869808 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31b97f-9cbb-45c1-b49b-b1b4c977c445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1715333032634911bcf0f7a41fab587a, user_id=90e3f9b6511347a692bed976865d5fad, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:21.872742 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31b97f-9cbb-45c1-b49b-b1b4c977c445 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:21.969102 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1168/2309] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:51:21 2022] PUT /identity/v3/projects/1715333032634911bcf0f7a41fab587a/users/90e3f9b6511347a692bed976865d5fad/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:21.974123 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984dda6-d2ce-4bbb-8266-d95d7b100a64 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:21.974907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984dda6-d2ce-4bbb-8266-d95d7b100a64 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:21.975185 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984dda6-d2ce-4bbb-8266-d95d7b100a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:21.995579 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-2984dda6-d2ce-4bbb-8266-d95d7b100a64 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:21.996813 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2984dda6-d2ce-4bbb-8266-d95d7b100a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:22.054538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1142/2310] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:22.064096 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31b97f-9cbb-45c1-b49b-b1b4c977c445 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:22.065995 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4a31b97f-9cbb-45c1-b49b-b1b4c977c445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:22.087982 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:22.124803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31b97f-9cbb-45c1-b49b-b1b4c977c445 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:22.128529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba148260-5d78-44ec-a452-f80857b0b6ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:22.129136 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba148260-5d78-44ec-a452-f80857b0b6ec service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:22.129420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba148260-5d78-44ec-a452-f80857b0b6ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:22.129696 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba148260-5d78-44ec-a452-f80857b0b6ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:22.139864 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba148260-5d78-44ec-a452-f80857b0b6ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:22.141326 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba148260-5d78-44ec-a452-f80857b0b6ec service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:22.151473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1169/2311] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:27.762661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7425c43-02e2-44f8-8416-ce62ef39648e None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:27.763184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7425c43-02e2-44f8-8416-ce62ef39648e None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:27.763316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7425c43-02e2-44f8-8416-ce62ef39648e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:27.791690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-e7425c43-02e2-44f8-8416-ce62ef39648e None None] MFA Rules not processed for user `90e3f9b6511347a692bed976865d5fad`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:27.792927 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e7425c43-02e2-44f8-8416-ce62ef39648e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:27.811343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:27.871514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1143/2312] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:27.876695 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba148260-5d78-44ec-a452-f80857b0b6ec service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:27.888642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba148260-5d78-44ec-a452-f80857b0b6ec service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:27.896810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83f701-4e19-4f2c-bf43-ac2cdba14565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:27.897685 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83f701-4e19-4f2c-bf43-ac2cdba14565 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:27.898881 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83f701-4e19-4f2c-bf43-ac2cdba14565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:27.899373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83f701-4e19-4f2c-bf43-ac2cdba14565 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:27.900225 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83f701-4e19-4f2c-bf43-ac2cdba14565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:27.909481 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83f701-4e19-4f2c-bf43-ac2cdba14565 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:28.009908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1170/2313] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:51:27 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:28.014132 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7425c43-02e2-44f8-8416-ce62ef39648e None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:28.025508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7425c43-02e2-44f8-8416-ce62ef39648e None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:28.085093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de46de-a490-4e5c-b28b-592a37a8e6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:28.085719 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de46de-a490-4e5c-b28b-592a37a8e6a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:28.085999 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de46de-a490-4e5c-b28b-592a37a8e6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:28.086270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de46de-a490-4e5c-b28b-592a37a8e6a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:28.087039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de46de-a490-4e5c-b28b-592a37a8e6a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:28.089154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de46de-a490-4e5c-b28b-592a37a8e6a0 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:28.162993 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1144/2314] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:51:28 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:51:28.163336 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83f701-4e19-4f2c-bf43-ac2cdba14565 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:28.174258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83f701-4e19-4f2c-bf43-ac2cdba14565 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:28.179541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71f933-626a-45be-b175-16103740bf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:28.180162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71f933-626a-45be-b175-16103740bf86 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:28.180477 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71f933-626a-45be-b175-16103740bf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:28.180756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71f933-626a-45be-b175-16103740bf86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:28.181411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa71f933-626a-45be-b175-16103740bf86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:28.183022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa71f933-626a-45be-b175-16103740bf86 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:28.196550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1171/2315] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:28.200924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de46de-a490-4e5c-b28b-592a37a8e6a0 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:28.211316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de46de-a490-4e5c-b28b-592a37a8e6a0 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:28.216237 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9213dc-efdb-45bc-8f89-e653e4423017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:28.216922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9213dc-efdb-45bc-8f89-e653e4423017 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:28.217201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9213dc-efdb-45bc-8f89-e653e4423017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:28.217470 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9213dc-efdb-45bc-8f89-e653e4423017 admin admin] PATH_INFO: `/v3/projects/032ab101adb94ba0921407b40b6beeea/users/40426d2d806f4648a107abb2393b8d01/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:28.237820 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9213dc-efdb-45bc-8f89-e653e4423017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=032ab101adb94ba0921407b40b6beeea, user_id=40426d2d806f4648a107abb2393b8d01, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:28.241861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9213dc-efdb-45bc-8f89-e653e4423017 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:28.338159 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1145/2316] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:51:28 2022] PUT /identity/v3/projects/032ab101adb94ba0921407b40b6beeea/users/40426d2d806f4648a107abb2393b8d01/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:28.342284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71f933-626a-45be-b175-16103740bf86 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:28.353564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71f933-626a-45be-b175-16103740bf86 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:28.428649 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1770d56-7940-403a-8eb9-0823aa3f728f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:28.429257 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1770d56-7940-403a-8eb9-0823aa3f728f admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:28.429609 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1770d56-7940-403a-8eb9-0823aa3f728f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:28.429791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1770d56-7940-403a-8eb9-0823aa3f728f admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:28.430430 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1770d56-7940-403a-8eb9-0823aa3f728f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:28.431998 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1770d56-7940-403a-8eb9-0823aa3f728f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:28.444725 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1172/2317] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:28.451305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9213dc-efdb-45bc-8f89-e653e4423017 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:28.460349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9213dc-efdb-45bc-8f89-e653e4423017 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:28.465364 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f8ba9-0c5a-4149-9ca3-0c8f4276e2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:28.466094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f8ba9-0c5a-4149-9ca3-0c8f4276e2c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:28.466435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f8ba9-0c5a-4149-9ca3-0c8f4276e2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:28.466730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f8ba9-0c5a-4149-9ca3-0c8f4276e2c7 admin admin] PATH_INFO: `/v3/projects/032ab101adb94ba0921407b40b6beeea/users/40426d2d806f4648a107abb2393b8d01/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:28.468959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f8ba9-0c5a-4149-9ca3-0c8f4276e2c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=032ab101adb94ba0921407b40b6beeea, user_id=40426d2d806f4648a107abb2393b8d01, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:28.471786 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f8ba9-0c5a-4149-9ca3-0c8f4276e2c7 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:28.482902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1146/2318] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:51:28 2022] PUT /identity/v3/projects/032ab101adb94ba0921407b40b6beeea/users/40426d2d806f4648a107abb2393b8d01/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:34.478032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ddb33-2238-4910-9df4-7c66db3d66bf None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:34.478032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ddb33-2238-4910-9df4-7c66db3d66bf None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:34.478032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ddb33-2238-4910-9df4-7c66db3d66bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:34.507634 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-509ddb33-2238-4910-9df4-7c66db3d66bf None None] MFA Rules not processed for user `40426d2d806f4648a107abb2393b8d01`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:34.508899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-509ddb33-2238-4910-9df4-7c66db3d66bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:34.605463 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1173/2319] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:34.619818 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f8ba9-0c5a-4149-9ca3-0c8f4276e2c7 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:34.622197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d54f8ba9-0c5a-4149-9ca3-0c8f4276e2c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:34.711513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f8ba9-0c5a-4149-9ca3-0c8f4276e2c7 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:34.718385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932be9a7-6c34-42f9-be28-e5c4a695ba6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:34.720034 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932be9a7-6c34-42f9-be28-e5c4a695ba6c service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:34.720321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932be9a7-6c34-42f9-be28-e5c4a695ba6c service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:34.720633 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932be9a7-6c34-42f9-be28-e5c4a695ba6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:34.728752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:34.790370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932be9a7-6c34-42f9-be28-e5c4a695ba6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:34.792243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932be9a7-6c34-42f9-be28-e5c4a695ba6c service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:34.805519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1147/2320] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:37.590693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616a42c-4e0b-4eb8-a0da-ba9ffd933c96 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:37.591211 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616a42c-4e0b-4eb8-a0da-ba9ffd933c96 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:37.591329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616a42c-4e0b-4eb8-a0da-ba9ffd933c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:37.610371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c616a42c-4e0b-4eb8-a0da-ba9ffd933c96 None None] MFA Rules not processed for user `90e3f9b6511347a692bed976865d5fad`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:37.612018 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c616a42c-4e0b-4eb8-a0da-ba9ffd933c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:37.619789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1174/2321] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:44.818465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=24 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:47.604679 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:48.073251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932be9a7-6c34-42f9-be28-e5c4a695ba6c service nova] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:48.074830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-932be9a7-6c34-42f9-be28-e5c4a695ba6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:48.139757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932be9a7-6c34-42f9-be28-e5c4a695ba6c service nova] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:48.144153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d22b8-71f1-4c50-8044-929f7ca1387d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:48.144804 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d22b8-71f1-4c50-8044-929f7ca1387d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:48.145085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d22b8-71f1-4c50-8044-929f7ca1387d admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:48.145358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d22b8-71f1-4c50-8044-929f7ca1387d admin admin] PATH_INFO: `/v3/users/78486adda1424457b0c6d5d91e69403e` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:48.146467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d22b8-71f1-4c50-8044-929f7ca1387d admin admin] RBAC: Authorizing `identity:delete_user(user_id=78486adda1424457b0c6d5d91e69403e)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:48.148917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d22b8-71f1-4c50-8044-929f7ca1387d admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:48.344280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a17d22b8-71f1-4c50-8044-929f7ca1387d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78486adda1424457b0c6d5d91e69403e'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:48.344952 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a17d22b8-71f1-4c50-8044-929f7ca1387d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78486adda1424457b0c6d5d91e69403e'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:48.345558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a17d22b8-71f1-4c50-8044-929f7ca1387d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78486adda1424457b0c6d5d91e69403e'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:48.352421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a17d22b8-71f1-4c50-8044-929f7ca1387d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78486adda1424457b0c6d5d91e69403e'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:48.409375 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a17d22b8-71f1-4c50-8044-929f7ca1387d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78486adda1424457b0c6d5d91e69403e'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:48.427692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1148/2322] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:51:48 2022] DELETE /identity/v3/users/78486adda1424457b0c6d5d91e69403e => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:48.432037 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321be3c-e6ed-42ea-8f1b-1dc0687c5b2e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:48.432782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321be3c-e6ed-42ea-8f1b-1dc0687c5b2e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:48.432782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321be3c-e6ed-42ea-8f1b-1dc0687c5b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:48.451891 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0321be3c-e6ed-42ea-8f1b-1dc0687c5b2e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:48.453125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0321be3c-e6ed-42ea-8f1b-1dc0687c5b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:48.510895 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1175/2323] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:48.520648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d22b8-71f1-4c50-8044-929f7ca1387d admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:48.522705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a17d22b8-71f1-4c50-8044-929f7ca1387d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:48.589415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d22b8-71f1-4c50-8044-929f7ca1387d admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:48.592979 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed8c00-981c-4765-8506-bfbb4fb53595 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:48.593581 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed8c00-981c-4765-8506-bfbb4fb53595 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:48.593859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed8c00-981c-4765-8506-bfbb4fb53595 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:48.594315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed8c00-981c-4765-8506-bfbb4fb53595 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:48.603496 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed8c00-981c-4765-8506-bfbb4fb53595 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:48.604947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed8c00-981c-4765-8506-bfbb4fb53595 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:48.614253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1149/2324] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:49.016974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321be3c-e6ed-42ea-8f1b-1dc0687c5b2e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:49.018553 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0321be3c-e6ed-42ea-8f1b-1dc0687c5b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:49.030974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321be3c-e6ed-42ea-8f1b-1dc0687c5b2e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:49.038170 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ccad1-14ca-44c4-979b-87a502e7dfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:49.038170 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ccad1-14ca-44c4-979b-87a502e7dfa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:49.038170 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ccad1-14ca-44c4-979b-87a502e7dfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:49.038170 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ccad1-14ca-44c4-979b-87a502e7dfa8 admin admin] PATH_INFO: `/v3/projects/d08617719a5947ddbbf208f4dcdde5e6` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:49.038728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ccad1-14ca-44c4-979b-87a502e7dfa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d08617719a5947ddbbf208f4dcdde5e6)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:49.040691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ccad1-14ca-44c4-979b-87a502e7dfa8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:49.148817 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a26ccad1-14ca-44c4-979b-87a502e7dfa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd08617719a5947ddbbf208f4dcdde5e6', 'request_id': 'req-a26ccad1-14ca-44c4-979b-87a502e7dfa8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:49.156314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a26ccad1-14ca-44c4-979b-87a502e7dfa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd08617719a5947ddbbf208f4dcdde5e6', 'request_id': 'req-a26ccad1-14ca-44c4-979b-87a502e7dfa8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:49.162166 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a26ccad1-14ca-44c4-979b-87a502e7dfa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd08617719a5947ddbbf208f4dcdde5e6', 'request_id': 'req-a26ccad1-14ca-44c4-979b-87a502e7dfa8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:49.229832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a26ccad1-14ca-44c4-979b-87a502e7dfa8 admin admin] The token cache is being invalidate because project d08617719a5947ddbbf208f4dcdde5e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:51:49.230048 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a26ccad1-14ca-44c4-979b-87a502e7dfa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:49.230646 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a26ccad1-14ca-44c4-979b-87a502e7dfa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:49.232296 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1176/2325] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:51:49 2022] DELETE /identity/v3/projects/d08617719a5947ddbbf208f4dcdde5e6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:49.237100 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320c82e-fab6-4370-9428-8fd1067d0cd7 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:49.237547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320c82e-fab6-4370-9428-8fd1067d0cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:49.237656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320c82e-fab6-4370-9428-8fd1067d0cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:49.258561 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-6320c82e-fab6-4370-9428-8fd1067d0cd7 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:49.259797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6320c82e-fab6-4370-9428-8fd1067d0cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:49.278682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:49.321383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1150/2326] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:49.329263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1342b-921d-4b8c-98f4-59eabce6a039 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:49.329548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1342b-921d-4b8c-98f4-59eabce6a039 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:49.329820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1342b-921d-4b8c-98f4-59eabce6a039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:49.351430 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3ca1342b-921d-4b8c-98f4-59eabce6a039 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:49.352692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3ca1342b-921d-4b8c-98f4-59eabce6a039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:49.361132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1177/2327] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:49.365498 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320c82e-fab6-4370-9428-8fd1067d0cd7 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:49.375560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320c82e-fab6-4370-9428-8fd1067d0cd7 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:49.380784 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d32f2-da2b-4d12-b7ef-7d3ebc662cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:49.381702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d32f2-da2b-4d12-b7ef-7d3ebc662cba admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:49.381702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d32f2-da2b-4d12-b7ef-7d3ebc662cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:49.381951 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d32f2-da2b-4d12-b7ef-7d3ebc662cba admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:49.382565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d32f2-da2b-4d12-b7ef-7d3ebc662cba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:51:49.382866 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d32f2-da2b-4d12-b7ef-7d3ebc662cba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:49.384539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d32f2-da2b-4d12-b7ef-7d3ebc662cba admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:49.395821 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1151/2328] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:51:49.400674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1342b-921d-4b8c-98f4-59eabce6a039 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:49.410028 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1342b-921d-4b8c-98f4-59eabce6a039 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:49.415036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45ba27-42a5-40da-a407-adb7b95f0579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:49.415623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45ba27-42a5-40da-a407-adb7b95f0579 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:49.415904 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45ba27-42a5-40da-a407-adb7b95f0579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:49.416180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45ba27-42a5-40da-a407-adb7b95f0579 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:49.416853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45ba27-42a5-40da-a407-adb7b95f0579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:49.418736 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45ba27-42a5-40da-a407-adb7b95f0579 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:49.512348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1178/2329] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:51:49 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:49.516313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d32f2-da2b-4d12-b7ef-7d3ebc662cba admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:49.526949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d32f2-da2b-4d12-b7ef-7d3ebc662cba admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:49.583095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b7acf-b2c9-486a-8c78-24962bcf43d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:49.585038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b7acf-b2c9-486a-8c78-24962bcf43d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:49.585038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b7acf-b2c9-486a-8c78-24962bcf43d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:49.585038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b7acf-b2c9-486a-8c78-24962bcf43d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:49.585412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b7acf-b2c9-486a-8c78-24962bcf43d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:49.587358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b7acf-b2c9-486a-8c78-24962bcf43d4 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:49.667905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1152/2330] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:51:49 2022] POST /identity/v3/users => generated 283 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:49.672100 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45ba27-42a5-40da-a407-adb7b95f0579 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:49.682441 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45ba27-42a5-40da-a407-adb7b95f0579 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:49.687059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31782636-1af9-41e1-ab32-5cad375d41c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:49.687657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31782636-1af9-41e1-ab32-5cad375d41c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:49.687936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31782636-1af9-41e1-ab32-5cad375d41c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:49.688260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31782636-1af9-41e1-ab32-5cad375d41c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:49.688935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31782636-1af9-41e1-ab32-5cad375d41c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:49.690522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31782636-1af9-41e1-ab32-5cad375d41c3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:49.701417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1179/2331] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:49.705461 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b7acf-b2c9-486a-8c78-24962bcf43d4 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:49.716057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b7acf-b2c9-486a-8c78-24962bcf43d4 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:49.721024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b141bb-f24d-493d-a6e9-d26e72a59d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:49.721628 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b141bb-f24d-493d-a6e9-d26e72a59d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:49.721928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b141bb-f24d-493d-a6e9-d26e72a59d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:49.722206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b141bb-f24d-493d-a6e9-d26e72a59d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:49.722933 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b141bb-f24d-493d-a6e9-d26e72a59d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:49.724557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b141bb-f24d-493d-a6e9-d26e72a59d4a admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:49.741252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1153/2332] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:51:49.742486 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31782636-1af9-41e1-ab32-5cad375d41c3 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:49.752849 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31782636-1af9-41e1-ab32-5cad375d41c3 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:49.758034 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dac8e-fd1f-4b95-9573-8d5535fdb328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:49.762994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dac8e-fd1f-4b95-9573-8d5535fdb328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:49.762994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dac8e-fd1f-4b95-9573-8d5535fdb328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:49.762994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dac8e-fd1f-4b95-9573-8d5535fdb328 admin admin] PATH_INFO: `/v3/projects/f41485c35d694c5da08637bb16df4dd6/users/c4d6d1f990c744e6b5aa923fa5a0ef8d/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:49.774621 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dac8e-fd1f-4b95-9573-8d5535fdb328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f41485c35d694c5da08637bb16df4dd6, user_id=c4d6d1f990c744e6b5aa923fa5a0ef8d, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:49.777315 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dac8e-fd1f-4b95-9573-8d5535fdb328 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:49.852352 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1180/2333] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:51:49 2022] PUT /identity/v3/projects/f41485c35d694c5da08637bb16df4dd6/users/c4d6d1f990c744e6b5aa923fa5a0ef8d/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:49.862944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7e93c-c488-4ae3-b3b5-755c8cf2121e None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:49.863231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7e93c-c488-4ae3-b3b5-755c8cf2121e None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:49.863505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7e93c-c488-4ae3-b3b5-755c8cf2121e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:49.891198 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-0fc7e93c-c488-4ae3-b3b5-755c8cf2121e None None] MFA Rules not processed for user `c4d6d1f990c744e6b5aa923fa5a0ef8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:49.892389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-0fc7e93c-c488-4ae3-b3b5-755c8cf2121e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:49.956898 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1154/2334] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:51:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:49.968485 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dac8e-fd1f-4b95-9573-8d5535fdb328 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:49.970142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-389dac8e-fd1f-4b95-9573-8d5535fdb328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:49.991753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:50.029455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dac8e-fd1f-4b95-9573-8d5535fdb328 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:50.033235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921874e9-2e28-47bb-ac15-761bb762c006 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:50.033854 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921874e9-2e28-47bb-ac15-761bb762c006 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:50.034133 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921874e9-2e28-47bb-ac15-761bb762c006 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:50.034409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921874e9-2e28-47bb-ac15-761bb762c006 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:50.042804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921874e9-2e28-47bb-ac15-761bb762c006 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:50.044215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921874e9-2e28-47bb-ac15-761bb762c006 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:50.052949 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1181/2335] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:59.953925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:00.151769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=17 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:38.756785 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c67dc3-b6d9-4e0e-8296-de64c5e6b8fe None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:38.757295 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c67dc3-b6d9-4e0e-8296-de64c5e6b8fe None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:38.757404 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c67dc3-b6d9-4e0e-8296-de64c5e6b8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:38.777379 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-22c67dc3-b6d9-4e0e-8296-de64c5e6b8fe None None] MFA Rules not processed for user `88efd5eb09064aa2aa89e6d9580c6476`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:38.778607 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-22c67dc3-b6d9-4e0e-8296-de64c5e6b8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:38.842282 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1155/2336] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:52:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:43.370749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921874e9-2e28-47bb-ac15-761bb762c006 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:43.372303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-921874e9-2e28-47bb-ac15-761bb762c006 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:43.428978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921874e9-2e28-47bb-ac15-761bb762c006 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:43.432539 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21258505-0dca-4803-b832-81d75911b253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:43.433123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21258505-0dca-4803-b832-81d75911b253 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:43.433406 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21258505-0dca-4803-b832-81d75911b253 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:43.433679 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21258505-0dca-4803-b832-81d75911b253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:43.435409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-21258505-0dca-4803-b832-81d75911b253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:43.496622 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21258505-0dca-4803-b832-81d75911b253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:43.498065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21258505-0dca-4803-b832-81d75911b253 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:43.507621 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1182/2337] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:48.834739 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=25 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:53.504608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=50 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:54.321833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c67dc3-b6d9-4e0e-8296-de64c5e6b8fe None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:54.393747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c67dc3-b6d9-4e0e-8296-de64c5e6b8fe None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:54.403201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373336f0-f953-4e9b-bde7-128951743a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:54.404472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373336f0-f953-4e9b-bde7-128951743a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:54.404771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373336f0-f953-4e9b-bde7-128951743a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:54.405045 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373336f0-f953-4e9b-bde7-128951743a75 admin admin] PATH_INFO: `/v3/users/c4d6d1f990c744e6b5aa923fa5a0ef8d` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:54.406652 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373336f0-f953-4e9b-bde7-128951743a75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4d6d1f990c744e6b5aa923fa5a0ef8d)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:54.408651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373336f0-f953-4e9b-bde7-128951743a75 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:54.633148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-373336f0-f953-4e9b-bde7-128951743a75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4d6d1f990c744e6b5aa923fa5a0ef8d'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:54.633996 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-373336f0-f953-4e9b-bde7-128951743a75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4d6d1f990c744e6b5aa923fa5a0ef8d'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:54.634681 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-373336f0-f953-4e9b-bde7-128951743a75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4d6d1f990c744e6b5aa923fa5a0ef8d'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:54.643146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-373336f0-f953-4e9b-bde7-128951743a75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4d6d1f990c744e6b5aa923fa5a0ef8d'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:54.729942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-373336f0-f953-4e9b-bde7-128951743a75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4d6d1f990c744e6b5aa923fa5a0ef8d'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:54.748960 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1156/2338] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:54 2022] DELETE /identity/v3/users/c4d6d1f990c744e6b5aa923fa5a0ef8d => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:54.753752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934e189-cda7-4cdd-aa94-864bab71e882 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:54.754172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934e189-cda7-4cdd-aa94-864bab71e882 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:54.754309 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934e189-cda7-4cdd-aa94-864bab71e882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:54.773615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e934e189-cda7-4cdd-aa94-864bab71e882 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:54.774807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e934e189-cda7-4cdd-aa94-864bab71e882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:54.833503 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1183/2339] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:54.843894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373336f0-f953-4e9b-bde7-128951743a75 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:54.852933 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-373336f0-f953-4e9b-bde7-128951743a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:54.903266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373336f0-f953-4e9b-bde7-128951743a75 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:54.906854 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bef1c-7ced-4074-b77f-40b29d149c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:54.907457 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bef1c-7ced-4074-b77f-40b29d149c6e service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:54.907743 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bef1c-7ced-4074-b77f-40b29d149c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:54.908014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bef1c-7ced-4074-b77f-40b29d149c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:54.917766 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bef1c-7ced-4074-b77f-40b29d149c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:54.919222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bef1c-7ced-4074-b77f-40b29d149c6e service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:54.928935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1157/2340] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:55.137289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512d71b-71c0-48f4-ae76-94ec357e51e5 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:55.137289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512d71b-71c0-48f4-ae76-94ec357e51e5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:55.137289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512d71b-71c0-48f4-ae76-94ec357e51e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:55.156380 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4512d71b-71c0-48f4-ae76-94ec357e51e5 None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:55.157539 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4512d71b-71c0-48f4-ae76-94ec357e51e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:55.217819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1184/2341] 10.222.0.70 () {58 vars in 1009 bytes} [Wed Aug 24 16:52:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:55.351274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bef1c-7ced-4074-b77f-40b29d149c6e service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:55.352774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-4f7bef1c-7ced-4074-b77f-40b29d149c6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:55.379177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:55.408870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512d71b-71c0-48f4-ae76-94ec357e51e5 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:55.408870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4512d71b-71c0-48f4-ae76-94ec357e51e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:55.425371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bef1c-7ced-4074-b77f-40b29d149c6e service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:55.431166 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a35be1-e85e-466a-b2df-eb6ac7c7bb41 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:55.431794 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a35be1-e85e-466a-b2df-eb6ac7c7bb41 service swift] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:55.432740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a35be1-e85e-466a-b2df-eb6ac7c7bb41 service swift] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:55.440026 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512d71b-71c0-48f4-ae76-94ec357e51e5 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:55.440648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a35be1-e85e-466a-b2df-eb6ac7c7bb41 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:55.441172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd2031-b06c-4b93-8377-8e209d638ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:55.441172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd2031-b06c-4b93-8377-8e209d638ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:55.441172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd2031-b06c-4b93-8377-8e209d638ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:55.441172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd2031-b06c-4b93-8377-8e209d638ef4 admin admin] PATH_INFO: `/v3/projects/f41485c35d694c5da08637bb16df4dd6` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:55.441785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd2031-b06c-4b93-8377-8e209d638ef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f41485c35d694c5da08637bb16df4dd6)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:55.441785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd2031-b06c-4b93-8377-8e209d638ef4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:55.443526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a35be1-e85e-466a-b2df-eb6ac7c7bb41 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:55.445248 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a35be1-e85e-466a-b2df-eb6ac7c7bb41 service swift] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:55.453682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1158/2342] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 16:52:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:52:55.574526 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-63cd2031-b06c-4b93-8377-8e209d638ef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f41485c35d694c5da08637bb16df4dd6', 'request_id': 'req-63cd2031-b06c-4b93-8377-8e209d638ef4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:55.583578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-63cd2031-b06c-4b93-8377-8e209d638ef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f41485c35d694c5da08637bb16df4dd6', 'request_id': 'req-63cd2031-b06c-4b93-8377-8e209d638ef4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:55.593362 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-63cd2031-b06c-4b93-8377-8e209d638ef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f41485c35d694c5da08637bb16df4dd6', 'request_id': 'req-63cd2031-b06c-4b93-8377-8e209d638ef4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:55.658439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-63cd2031-b06c-4b93-8377-8e209d638ef4 admin admin] The token cache is being invalidate because project f41485c35d694c5da08637bb16df4dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:55.658738 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-63cd2031-b06c-4b93-8377-8e209d638ef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:55.659470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-63cd2031-b06c-4b93-8377-8e209d638ef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:55.664559 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1185/2343] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:55 2022] DELETE /identity/v3/projects/f41485c35d694c5da08637bb16df4dd6 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:52:55.667138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bb57c-9ace-4895-9a3c-6f8432440801 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:55.667557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bb57c-9ace-4895-9a3c-6f8432440801 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:55.667952 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bb57c-9ace-4895-9a3c-6f8432440801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:55.692521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-d61bb57c-9ace-4895-9a3c-6f8432440801 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:55.694053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d61bb57c-9ace-4895-9a3c-6f8432440801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:55.767554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1159/2344] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:55.782052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941f1d1-d10a-4b55-bba8-7f8baac1bd7b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:55.782515 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941f1d1-d10a-4b55-bba8-7f8baac1bd7b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:55.782653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941f1d1-d10a-4b55-bba8-7f8baac1bd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:55.807662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b941f1d1-d10a-4b55-bba8-7f8baac1bd7b None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:55.809244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b941f1d1-d10a-4b55-bba8-7f8baac1bd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:55.819925 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1186/2345] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:55.824436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bb57c-9ace-4895-9a3c-6f8432440801 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:55.835028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bb57c-9ace-4895-9a3c-6f8432440801 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:55.851952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941f1d1-d10a-4b55-bba8-7f8baac1bd7b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:55.852376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffeb10-98fd-41d5-9817-9644b5a11cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:55.853083 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffeb10-98fd-41d5-9817-9644b5a11cfb admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:55.853361 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffeb10-98fd-41d5-9817-9644b5a11cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:55.853630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffeb10-98fd-41d5-9817-9644b5a11cfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:55.854531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b941f1d1-d10a-4b55-bba8-7f8baac1bd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:55.855335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffeb10-98fd-41d5-9817-9644b5a11cfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:52:55.855638 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffeb10-98fd-41d5-9817-9644b5a11cfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:55.868921 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffeb10-98fd-41d5-9817-9644b5a11cfb admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:55.887472 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:55.892100 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1160/2346] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:52:55.905272 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffeb10-98fd-41d5-9817-9644b5a11cfb admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:55.920266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffeb10-98fd-41d5-9817-9644b5a11cfb admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:55.927332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f819913-4549-47cf-9116-2b53e42bf76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:55.927502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941f1d1-d10a-4b55-bba8-7f8baac1bd7b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:55.929142 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f819913-4549-47cf-9116-2b53e42bf76d admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:55.929142 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f819913-4549-47cf-9116-2b53e42bf76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:55.929142 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f819913-4549-47cf-9116-2b53e42bf76d admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:55.929849 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f819913-4549-47cf-9116-2b53e42bf76d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:55.931854 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f819913-4549-47cf-9116-2b53e42bf76d admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:55.932010 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e790e06-6a19-4ff7-9c35-ed88071e9903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:55.932790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e790e06-6a19-4ff7-9c35-ed88071e9903 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:55.933158 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e790e06-6a19-4ff7-9c35-ed88071e9903 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:55.933574 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e790e06-6a19-4ff7-9c35-ed88071e9903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:55.935869 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7e790e06-6a19-4ff7-9c35-ed88071e9903 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:56.036190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e790e06-6a19-4ff7-9c35-ed88071e9903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:56.038123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e790e06-6a19-4ff7-9c35-ed88071e9903 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:56.040619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1161/2347] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:55 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:52:56.047651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f819913-4549-47cf-9116-2b53e42bf76d admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:56.055260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1187/2348] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:52:56.173738 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f819913-4549-47cf-9116-2b53e42bf76d admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:56.190518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8189df-a348-4f26-a56f-ba639cd98172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:56.191723 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8189df-a348-4f26-a56f-ba639cd98172 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:56.192458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8189df-a348-4f26-a56f-ba639cd98172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:56.192849 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8189df-a348-4f26-a56f-ba639cd98172 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:56.194066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8189df-a348-4f26-a56f-ba639cd98172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:56.196228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8189df-a348-4f26-a56f-ba639cd98172 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:56.428762 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1162/2349] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:56 2022] POST /identity/v3/users => generated 287 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:56.433738 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e790e06-6a19-4ff7-9c35-ed88071e9903 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:56.445600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e790e06-6a19-4ff7-9c35-ed88071e9903 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:56.451747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03d217-bf9b-44c4-9455-eed83f2a4f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:56.452363 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03d217-bf9b-44c4-9455-eed83f2a4f43 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:56.452683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03d217-bf9b-44c4-9455-eed83f2a4f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:56.452991 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03d217-bf9b-44c4-9455-eed83f2a4f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:56.453876 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03d217-bf9b-44c4-9455-eed83f2a4f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:56.456195 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03d217-bf9b-44c4-9455-eed83f2a4f43 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:56.471304 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1188/2350] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:52:56.472711 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8189df-a348-4f26-a56f-ba639cd98172 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:56.484129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8189df-a348-4f26-a56f-ba639cd98172 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:56.488997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae986b3f-7b11-4d90-827c-7c94f134ccfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:56.489677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae986b3f-7b11-4d90-827c-7c94f134ccfb admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:56.490042 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae986b3f-7b11-4d90-827c-7c94f134ccfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:56.490385 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae986b3f-7b11-4d90-827c-7c94f134ccfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:56.491109 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae986b3f-7b11-4d90-827c-7c94f134ccfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:56.492844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae986b3f-7b11-4d90-827c-7c94f134ccfb admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:56.494365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:56.505086 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1163/2351] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:56.513887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03d217-bf9b-44c4-9455-eed83f2a4f43 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:56.526428 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03d217-bf9b-44c4-9455-eed83f2a4f43 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:56.531550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6cdd4-100b-4703-a597-425986f5e2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:56.532304 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6cdd4-100b-4703-a597-425986f5e2f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:56.532725 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6cdd4-100b-4703-a597-425986f5e2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:56.533081 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6cdd4-100b-4703-a597-425986f5e2f4 admin admin] PATH_INFO: `/v3/projects/346b4dbf4f534634869415fb20cec650/users/86c9d9b34eb74f9abe366327d9493c87/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:56.552931 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6cdd4-100b-4703-a597-425986f5e2f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=346b4dbf4f534634869415fb20cec650, user_id=86c9d9b34eb74f9abe366327d9493c87, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:56.555619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6cdd4-100b-4703-a597-425986f5e2f4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:56.614502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1189/2352] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:56 2022] PUT /identity/v3/projects/346b4dbf4f534634869415fb20cec650/users/86c9d9b34eb74f9abe366327d9493c87/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:56.632484 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6164dd-041a-4027-8783-dc10e07c2556 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:56.632484 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6164dd-041a-4027-8783-dc10e07c2556 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:56.632484 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6164dd-041a-4027-8783-dc10e07c2556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:56.663841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-dd6164dd-041a-4027-8783-dc10e07c2556 None None] MFA Rules not processed for user `86c9d9b34eb74f9abe366327d9493c87`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:56.665117 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-dd6164dd-041a-4027-8783-dc10e07c2556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:56.741103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1164/2353] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:52:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:56.752179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6cdd4-100b-4703-a597-425986f5e2f4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:56.753989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a9d6cdd4-100b-4703-a597-425986f5e2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:56.824129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6cdd4-100b-4703-a597-425986f5e2f4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:56.829632 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069350c9-c577-4706-b6df-8380a8e0bbc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:56.830284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069350c9-c577-4706-b6df-8380a8e0bbc3 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:56.830589 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069350c9-c577-4706-b6df-8380a8e0bbc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:56.830865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069350c9-c577-4706-b6df-8380a8e0bbc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:56.843535 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069350c9-c577-4706-b6df-8380a8e0bbc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:56.845248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069350c9-c577-4706-b6df-8380a8e0bbc3 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:56.855417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1190/2354] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:57.443297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6164dd-041a-4027-8783-dc10e07c2556 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:57.550967 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6164dd-041a-4027-8783-dc10e07c2556 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:57.556580 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14a01a-e390-44d4-b1ab-180bbcbae64b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:57.558049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14a01a-e390-44d4-b1ab-180bbcbae64b service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:57.558843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14a01a-e390-44d4-b1ab-180bbcbae64b service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:57.559980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14a01a-e390-44d4-b1ab-180bbcbae64b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:57.565402 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1f14a01a-e390-44d4-b1ab-180bbcbae64b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:57.703392 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14a01a-e390-44d4-b1ab-180bbcbae64b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:57.705545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14a01a-e390-44d4-b1ab-180bbcbae64b service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:57.733789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1165/2355] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:06.850732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:07.726027 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=88 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:08.355777 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069350c9-c577-4706-b6df-8380a8e0bbc3 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:08.357316 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-069350c9-c577-4706-b6df-8380a8e0bbc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:08.373984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069350c9-c577-4706-b6df-8380a8e0bbc3 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:08.379720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874235f-d370-4aa6-b914-40973243fdb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:08.380589 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874235f-d370-4aa6-b914-40973243fdb0 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:08.380712 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874235f-d370-4aa6-b914-40973243fdb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:08.380949 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874235f-d370-4aa6-b914-40973243fdb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:08.382597 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d874235f-d370-4aa6-b914-40973243fdb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:08.444003 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874235f-d370-4aa6-b914-40973243fdb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:08.445821 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874235f-d370-4aa6-b914-40973243fdb0 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:08.457554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1191/2356] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:12.060015 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14a01a-e390-44d4-b1ab-180bbcbae64b service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:12.139230 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14a01a-e390-44d4-b1ab-180bbcbae64b service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:12.143991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa2288-8fd6-444b-b62c-afd73385954f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:12.144670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa2288-8fd6-444b-b62c-afd73385954f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:12.144953 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa2288-8fd6-444b-b62c-afd73385954f admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:12.145269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa2288-8fd6-444b-b62c-afd73385954f admin admin] PATH_INFO: `/v3/users/86c9d9b34eb74f9abe366327d9493c87` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:12.146575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa2288-8fd6-444b-b62c-afd73385954f admin admin] RBAC: Authorizing `identity:delete_user(user_id=86c9d9b34eb74f9abe366327d9493c87)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:12.152648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa2288-8fd6-444b-b62c-afd73385954f admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:12.332499 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c6fa2288-8fd6-444b-b62c-afd73385954f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86c9d9b34eb74f9abe366327d9493c87'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:12.333148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c6fa2288-8fd6-444b-b62c-afd73385954f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86c9d9b34eb74f9abe366327d9493c87'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:12.333746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c6fa2288-8fd6-444b-b62c-afd73385954f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86c9d9b34eb74f9abe366327d9493c87'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:12.341335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c6fa2288-8fd6-444b-b62c-afd73385954f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86c9d9b34eb74f9abe366327d9493c87'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:12.415590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c6fa2288-8fd6-444b-b62c-afd73385954f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86c9d9b34eb74f9abe366327d9493c87'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:12.435157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1166/2357] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:12 2022] DELETE /identity/v3/users/86c9d9b34eb74f9abe366327d9493c87 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:12.439460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d6f56-2cd0-467c-84f2-7f2ee438f75b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:12.440042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d6f56-2cd0-467c-84f2-7f2ee438f75b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:12.440042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d6f56-2cd0-467c-84f2-7f2ee438f75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:12.458599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-391d6f56-2cd0-467c-84f2-7f2ee438f75b None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:12.459798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-391d6f56-2cd0-467c-84f2-7f2ee438f75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:12.518636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1192/2358] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:12.529271 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa2288-8fd6-444b-b62c-afd73385954f admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:12.530884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c6fa2288-8fd6-444b-b62c-afd73385954f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:12.595043 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa2288-8fd6-444b-b62c-afd73385954f admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:12.598688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337e40b-f77d-4e1e-bd81-cd975d293c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:12.599383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337e40b-f77d-4e1e-bd81-cd975d293c15 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:12.599664 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337e40b-f77d-4e1e-bd81-cd975d293c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:12.599935 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337e40b-f77d-4e1e-bd81-cd975d293c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:12.613068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337e40b-f77d-4e1e-bd81-cd975d293c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:12.614640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337e40b-f77d-4e1e-bd81-cd975d293c15 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:12.624311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1167/2359] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:13.041119 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d6f56-2cd0-467c-84f2-7f2ee438f75b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:13.042920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-391d6f56-2cd0-467c-84f2-7f2ee438f75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:13.059333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d6f56-2cd0-467c-84f2-7f2ee438f75b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:13.064076 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a87e15-dee7-4794-9ff4-4674e6b49082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:13.068575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a87e15-dee7-4794-9ff4-4674e6b49082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:13.068575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a87e15-dee7-4794-9ff4-4674e6b49082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:13.068575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a87e15-dee7-4794-9ff4-4674e6b49082 admin admin] PATH_INFO: `/v3/projects/346b4dbf4f534634869415fb20cec650` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:13.068575 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a87e15-dee7-4794-9ff4-4674e6b49082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=346b4dbf4f534634869415fb20cec650)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:13.069363 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a87e15-dee7-4794-9ff4-4674e6b49082 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:13.174693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-51a87e15-dee7-4794-9ff4-4674e6b49082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '346b4dbf4f534634869415fb20cec650', 'request_id': 'req-51a87e15-dee7-4794-9ff4-4674e6b49082'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:13.180871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-51a87e15-dee7-4794-9ff4-4674e6b49082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '346b4dbf4f534634869415fb20cec650', 'request_id': 'req-51a87e15-dee7-4794-9ff4-4674e6b49082'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:13.186479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-51a87e15-dee7-4794-9ff4-4674e6b49082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '346b4dbf4f534634869415fb20cec650', 'request_id': 'req-51a87e15-dee7-4794-9ff4-4674e6b49082'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:13.282335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-51a87e15-dee7-4794-9ff4-4674e6b49082 admin admin] The token cache is being invalidate because project 346b4dbf4f534634869415fb20cec650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:13.282629 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-51a87e15-dee7-4794-9ff4-4674e6b49082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:13.283280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-51a87e15-dee7-4794-9ff4-4674e6b49082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:13.284819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1193/2360] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:13 2022] DELETE /identity/v3/projects/346b4dbf4f534634869415fb20cec650 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:13.459727 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337e40b-f77d-4e1e-bd81-cd975d293c15 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:13.461284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2337e40b-f77d-4e1e-bd81-cd975d293c15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:13.473753 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337e40b-f77d-4e1e-bd81-cd975d293c15 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:13.491955 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:13.522458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455749d-3759-4288-8afe-239c773b4948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:13.523120 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455749d-3759-4288-8afe-239c773b4948 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:13.523628 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455749d-3759-4288-8afe-239c773b4948 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:13.523962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455749d-3759-4288-8afe-239c773b4948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:13.526714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1455749d-3759-4288-8afe-239c773b4948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:13.535838 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455749d-3759-4288-8afe-239c773b4948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:13.537358 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455749d-3759-4288-8afe-239c773b4948 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:13.545490 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1168/2361] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:13.953188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3cfc4-7d47-4fef-98e7-02b92baafe47 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:13.953711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3cfc4-7d47-4fef-98e7-02b92baafe47 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:13.953833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3cfc4-7d47-4fef-98e7-02b92baafe47 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:13.955111 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1194/2362] 10.222.0.70 () {54 vars in 921 bytes} [Wed Aug 24 16:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:53:13.960930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0affee7-bacc-41b0-b977-4ff8b418ca12 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:13.961217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0affee7-bacc-41b0-b977-4ff8b418ca12 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:13.961523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0affee7-bacc-41b0-b977-4ff8b418ca12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:13.982288 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-a0affee7-bacc-41b0-b977-4ff8b418ca12 None None] MFA Rules not processed for user `e333fca7790244f6a4ebceabb55faba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:13.983518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a0affee7-bacc-41b0-b977-4ff8b418ca12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:14.055681 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1169/2363] 10.222.0.70 () {58 vars in 1018 bytes} [Wed Aug 24 16:53:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:14.067950 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3cfc4-7d47-4fef-98e7-02b92baafe47 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:14.069649 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-99c3cfc4-7d47-4fef-98e7-02b92baafe47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:14.095969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:14.127751 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3cfc4-7d47-4fef-98e7-02b92baafe47 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:14.131282 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0f050-fe9c-41fc-9162-564e2e0b86ae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56418ff737504181a59714aad9f1c375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:14.131881 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0f050-fe9c-41fc-9162-564e2e0b86ae service glance] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:14.132160 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0f050-fe9c-41fc-9162-564e2e0b86ae service glance] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:14.132461 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0f050-fe9c-41fc-9162-564e2e0b86ae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:14.142298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0f050-fe9c-41fc-9162-564e2e0b86ae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:14.143719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0f050-fe9c-41fc-9162-564e2e0b86ae service glance] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:14.154636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1195/2364] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:14.406064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b0891-630c-4e52-ba2e-66294e229e28 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:14.406348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b0891-630c-4e52-ba2e-66294e229e28 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:14.406621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b0891-630c-4e52-ba2e-66294e229e28 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:14.407963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1170/2365] 10.222.0.70 () {54 vars in 921 bytes} [Wed Aug 24 16:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:53:14.412935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eb82a-5b7f-47e1-bd2c-434e7ab35118 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:14.413226 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eb82a-5b7f-47e1-bd2c-434e7ab35118 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:14.413527 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eb82a-5b7f-47e1-bd2c-434e7ab35118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:14.433667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0b8eb82a-5b7f-47e1-bd2c-434e7ab35118 None None] MFA Rules not processed for user `942b54c39ae5403c8ae3bcca1c31bf1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:14.434870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0b8eb82a-5b7f-47e1-bd2c-434e7ab35118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:14.441313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1196/2366] 10.222.0.70 () {58 vars in 1018 bytes} [Wed Aug 24 16:53:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:14.452867 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b0891-630c-4e52-ba2e-66294e229e28 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:14.454657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6d0b0891-630c-4e52-ba2e-66294e229e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:14.468212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b0891-630c-4e52-ba2e-66294e229e28 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:14.471738 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac209278-b95a-4e8c-b733-a6cb48cc26c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:14.472323 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac209278-b95a-4e8c-b733-a6cb48cc26c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:14.472628 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac209278-b95a-4e8c-b733-a6cb48cc26c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:14.472898 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac209278-b95a-4e8c-b733-a6cb48cc26c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:14.480904 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac209278-b95a-4e8c-b733-a6cb48cc26c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:14.482315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac209278-b95a-4e8c-b733-a6cb48cc26c7 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:14.489971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1171/2367] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:14.571605 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e4a5c-8622-4bfd-9f61-b304fdf8b7e0 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:14.571894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e4a5c-8622-4bfd-9f61-b304fdf8b7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:14.572171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e4a5c-8622-4bfd-9f61-b304fdf8b7e0 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:14.573544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1197/2368] 10.222.0.70 () {54 vars in 921 bytes} [Wed Aug 24 16:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:53:14.578422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944cfc-61b4-4d5f-be84-7dba1826458c None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:14.578709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944cfc-61b4-4d5f-be84-7dba1826458c None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:14.578987 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944cfc-61b4-4d5f-be84-7dba1826458c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:14.598828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-c2944cfc-61b4-4d5f-be84-7dba1826458c None None] MFA Rules not processed for user `e333fca7790244f6a4ebceabb55faba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:14.600066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c2944cfc-61b4-4d5f-be84-7dba1826458c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:14.610094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1172/2369] 10.222.0.70 () {58 vars in 1018 bytes} [Wed Aug 24 16:53:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:14.621185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e4a5c-8622-4bfd-9f61-b304fdf8b7e0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:14.631210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-481e4a5c-8622-4bfd-9f61-b304fdf8b7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:14.636233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e4a5c-8622-4bfd-9f61-b304fdf8b7e0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:14.639827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dcd59-ce02-49e5-8c08-2f2bf0750b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:14.640425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dcd59-ce02-49e5-8c08-2f2bf0750b6a service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:14.640726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dcd59-ce02-49e5-8c08-2f2bf0750b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:14.640996 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dcd59-ce02-49e5-8c08-2f2bf0750b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:14.653486 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84dcd59-ce02-49e5-8c08-2f2bf0750b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:14.655648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84dcd59-ce02-49e5-8c08-2f2bf0750b6a service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:14.668981 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1198/2370] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:14.735681 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2944cfc-61b4-4d5f-be84-7dba1826458c None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:14.737095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c2944cfc-61b4-4d5f-be84-7dba1826458c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:14.803525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2944cfc-61b4-4d5f-be84-7dba1826458c None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:14.808474 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863c364-081a-4a7c-a790-d67c62a4a84f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:14.809070 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863c364-081a-4a7c-a790-d67c62a4a84f service placement] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:14.809346 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863c364-081a-4a7c-a790-d67c62a4a84f service placement] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:14.809614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863c364-081a-4a7c-a790-d67c62a4a84f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:14.811301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3863c364-081a-4a7c-a790-d67c62a4a84f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:14.822201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863c364-081a-4a7c-a790-d67c62a4a84f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:14.824264 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863c364-081a-4a7c-a790-d67c62a4a84f service placement] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:14.833823 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1173/2371] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:18.448287 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1d5c2-3498-4c0f-827c-08e58ae3b968 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:18.448769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1d5c2-3498-4c0f-827c-08e58ae3b968 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:18.448887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1d5c2-3498-4c0f-827c-08e58ae3b968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:18.467718 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-35c1d5c2-3498-4c0f-827c-08e58ae3b968 None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:18.468957 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-35c1d5c2-3498-4c0f-827c-08e58ae3b968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:18.523902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1199/2372] 10.222.0.70 () {58 vars in 1009 bytes} [Wed Aug 24 16:53:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:18.535490 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863c364-081a-4a7c-a790-d67c62a4a84f service placement] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:18.536916 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3863c364-081a-4a7c-a790-d67c62a4a84f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:18.571485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:18.608741 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863c364-081a-4a7c-a790-d67c62a4a84f service placement] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:18.613867 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2c8ad-ec4c-4482-ba0c-5d4726d21a6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:18.614472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2c8ad-ec4c-4482-ba0c-5d4726d21a6f service swift] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:18.614755 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2c8ad-ec4c-4482-ba0c-5d4726d21a6f service swift] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:18.615031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2c8ad-ec4c-4482-ba0c-5d4726d21a6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:18.624064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2c8ad-ec4c-4482-ba0c-5d4726d21a6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:18.625764 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2c8ad-ec4c-4482-ba0c-5d4726d21a6f service swift] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:18.634294 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1174/2373] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 16:53:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:53:28.517215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=50 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:28.659787 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=20 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:35.214289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1d5c2-3498-4c0f-827c-08e58ae3b968 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:35.215798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-35c1d5c2-3498-4c0f-827c-08e58ae3b968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:35.232030 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1d5c2-3498-4c0f-827c-08e58ae3b968 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:35.237719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2fcf0d-e0d0-421f-ab8b-177d3c68f4d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:35.238237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fcf0d-e0d0-421f-ab8b-177d3c68f4d7 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:35.238521 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fcf0d-e0d0-421f-ab8b-177d3c68f4d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:35.238800 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fcf0d-e0d0-421f-ab8b-177d3c68f4d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:35.248520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6c2fcf0d-e0d0-421f-ab8b-177d3c68f4d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:35.299914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2fcf0d-e0d0-421f-ab8b-177d3c68f4d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:35.301567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2fcf0d-e0d0-421f-ab8b-177d3c68f4d7 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:35.311086 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1200/2374] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:35 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:45.071548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2c8ad-ec4c-4482-ba0c-5d4726d21a6f service swift] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:45.081777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2c8ad-ec4c-4482-ba0c-5d4726d21a6f service swift] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:45.085720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd7fa3-b567-48e4-a37e-74d799ff60b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:45.086404 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd7fa3-b567-48e4-a37e-74d799ff60b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:45.087087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd7fa3-b567-48e4-a37e-74d799ff60b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:45.087087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd7fa3-b567-48e4-a37e-74d799ff60b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:45.088877 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-eedd7fa3-b567-48e4-a37e-74d799ff60b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:45.101045 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd7fa3-b567-48e4-a37e-74d799ff60b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:45.102502 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd7fa3-b567-48e4-a37e-74d799ff60b8 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:45.112395 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1175/2375] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:45.306377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=28 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:50.250835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c7fe1-48c6-4619-852f-d745dac277c8 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:50.251413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c7fe1-48c6-4619-852f-d745dac277c8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:50.251413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c7fe1-48c6-4619-852f-d745dac277c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:50.271236 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7d4c7fe1-48c6-4619-852f-d745dac277c8 None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:50.272479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7d4c7fe1-48c6-4619-852f-d745dac277c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:50.283361 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1201/2376] 10.222.0.70 () {58 vars in 967 bytes} [Wed Aug 24 16:53:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:50.296983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd7fa3-b567-48e4-a37e-74d799ff60b8 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:50.309357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd7fa3-b567-48e4-a37e-74d799ff60b8 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:50.314220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a5a84-5e96-433b-bb06-bcbccebeb86b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:50.314811 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a5a84-5e96-433b-bb06-bcbccebeb86b service swift] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:50.315181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a5a84-5e96-433b-bb06-bcbccebeb86b service swift] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:50.315460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a5a84-5e96-433b-bb06-bcbccebeb86b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:50.323301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a5a84-5e96-433b-bb06-bcbccebeb86b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:50.325559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a5a84-5e96-433b-bb06-bcbccebeb86b service swift] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:50.333444 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1176/2377] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 16:53:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:54:00.265852 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:00.330234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=12 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:06.433088 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dccaf4-19d8-42b0-bc28-98fa430c51a0 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:06.433612 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dccaf4-19d8-42b0-bc28-98fa430c51a0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:06.433730 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dccaf4-19d8-42b0-bc28-98fa430c51a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:06.460728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-46dccaf4-19d8-42b0-bc28-98fa430c51a0 None None] MFA Rules not processed for user `45913fc3d9ba42ea9e971db25e656c61`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:06.461914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-46dccaf4-19d8-42b0-bc28-98fa430c51a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:06.469656 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1202/2378] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:54:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:13.945578 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a5a84-5e96-433b-bb06-bcbccebeb86b service swift] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:13.959307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a5a84-5e96-433b-bb06-bcbccebeb86b service swift] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:13.964941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0222ff-3bc5-4dcc-9e4e-f72b24aa6795 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:13.965549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0222ff-3bc5-4dcc-9e4e-f72b24aa6795 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:13.965907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0222ff-3bc5-4dcc-9e4e-f72b24aa6795 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:13.966185 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0222ff-3bc5-4dcc-9e4e-f72b24aa6795 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:13.967987 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-dc0222ff-3bc5-4dcc-9e4e-f72b24aa6795 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:13.981355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0222ff-3bc5-4dcc-9e4e-f72b24aa6795 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:13.983049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0222ff-3bc5-4dcc-9e4e-f72b24aa6795 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:13.994557 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1177/2379] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:16.450738 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:23.989817 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=6 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:44.945774 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dccaf4-19d8-42b0-bc28-98fa430c51a0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:44.958916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dccaf4-19d8-42b0-bc28-98fa430c51a0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:44.964261 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa553140-1690-48a7-897f-56042a1f17af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:44.964889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa553140-1690-48a7-897f-56042a1f17af service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:44.965173 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa553140-1690-48a7-897f-56042a1f17af service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:44.965445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa553140-1690-48a7-897f-56042a1f17af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:44.967154 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fa553140-1690-48a7-897f-56042a1f17af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:45.024324 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa553140-1690-48a7-897f-56042a1f17af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:45.025922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa553140-1690-48a7-897f-56042a1f17af service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:45.034853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1203/2380] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:48.440204 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0222ff-3bc5-4dcc-9e4e-f72b24aa6795 service nova] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:48.451907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0222ff-3bc5-4dcc-9e4e-f72b24aa6795 service nova] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:48.457493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892a38e-d709-4769-a774-7675fb23f932 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:48.458099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892a38e-d709-4769-a774-7675fb23f932 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:48.458378 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892a38e-d709-4769-a774-7675fb23f932 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:48.458650 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892a38e-d709-4769-a774-7675fb23f932 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:48.460815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-9892a38e-d709-4769-a774-7675fb23f932 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:48.474763 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892a38e-d709-4769-a774-7675fb23f932 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:48.476469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892a38e-d709-4769-a774-7675fb23f932 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:48.488682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1178/2381] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:55.030658 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=28 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:58.483595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=6 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:00.749400 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c494e-ac99-4372-8654-6d975deb312c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:00.749855 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c494e-ac99-4372-8654-6d975deb312c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:00.750059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c494e-ac99-4372-8654-6d975deb312c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:00.769584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-2b5c494e-ac99-4372-8654-6d975deb312c None None] MFA Rules not processed for user `88efd5eb09064aa2aa89e6d9580c6476`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:00.770716 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2b5c494e-ac99-4372-8654-6d975deb312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:00.777549 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1204/2382] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:55:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:10.764289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:13.158859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892a38e-d709-4769-a774-7675fb23f932 service nova] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:13.173001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892a38e-d709-4769-a774-7675fb23f932 service nova] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:13.196499 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a9261-4eee-4a3e-8e33-c2386a352adf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:13.196499 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a9261-4eee-4a3e-8e33-c2386a352adf service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:13.196499 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a9261-4eee-4a3e-8e33-c2386a352adf service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:13.196499 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a9261-4eee-4a3e-8e33-c2386a352adf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:13.197194 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-745a9261-4eee-4a3e-8e33-c2386a352adf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:13.197194 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a9261-4eee-4a3e-8e33-c2386a352adf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:13.198226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a9261-4eee-4a3e-8e33-c2386a352adf service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:13.209437 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1179/2383] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:21.986665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843b891-dca2-4762-843a-6d0c79418ae8 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:21.987096 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843b891-dca2-4762-843a-6d0c79418ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:21.987243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843b891-dca2-4762-843a-6d0c79418ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:22.006978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5843b891-dca2-4762-843a-6d0c79418ae8 None None] MFA Rules not processed for user `7a930257a6584b04bae278aa83940264`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:22.008841 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5843b891-dca2-4762-843a-6d0c79418ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:22.063002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1205/2384] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:55:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:22.365851 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a9261-4eee-4a3e-8e33-c2386a352adf service nova] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:22.375780 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a9261-4eee-4a3e-8e33-c2386a352adf service nova] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:22.379589 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9de22-0707-4a8b-891e-ae4b51ac9efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:22.386253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9de22-0707-4a8b-891e-ae4b51ac9efd service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:22.386253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9de22-0707-4a8b-891e-ae4b51ac9efd service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:22.386253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9de22-0707-4a8b-891e-ae4b51ac9efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:22.386253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-f7b9de22-0707-4a8b-891e-ae4b51ac9efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:22.391642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9de22-0707-4a8b-891e-ae4b51ac9efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:22.393093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9de22-0707-4a8b-891e-ae4b51ac9efd service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:22.402107 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1180/2385] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:32.056261 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=25 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:32.397778 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=12 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:45.217203 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843b891-dca2-4762-843a-6d0c79418ae8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:45.228365 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843b891-dca2-4762-843a-6d0c79418ae8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:45.231901 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04e212-c186-4cc0-9db8-f7616db74105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:45.232529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04e212-c186-4cc0-9db8-f7616db74105 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:45.232807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04e212-c186-4cc0-9db8-f7616db74105 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:45.233074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04e212-c186-4cc0-9db8-f7616db74105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:45.234753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dd04e212-c186-4cc0-9db8-f7616db74105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:45.297519 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04e212-c186-4cc0-9db8-f7616db74105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:45.298949 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04e212-c186-4cc0-9db8-f7616db74105 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:45.308547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1206/2386] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:46.324923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07daa3f1-ee22-4d55-ac65-5d293f6687ec None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:46.325714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07daa3f1-ee22-4d55-ac65-5d293f6687ec None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:46.325714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07daa3f1-ee22-4d55-ac65-5d293f6687ec None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:46.327443 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1181/2387] 10.222.0.70 () {54 vars in 921 bytes} [Wed Aug 24 16:55:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:55:46.334715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39313a-ce12-4dfe-a053-8da18364da10 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:46.335309 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39313a-ce12-4dfe-a053-8da18364da10 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:46.335593 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39313a-ce12-4dfe-a053-8da18364da10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:46.360846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fe39313a-ce12-4dfe-a053-8da18364da10 None None] MFA Rules not processed for user `942b54c39ae5403c8ae3bcca1c31bf1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:46.364954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fe39313a-ce12-4dfe-a053-8da18364da10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:46.388673 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1207/2388] 10.222.0.70 () {58 vars in 1018 bytes} [Wed Aug 24 16:55:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:46.395582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07daa3f1-ee22-4d55-ac65-5d293f6687ec None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:46.416144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07daa3f1-ee22-4d55-ac65-5d293f6687ec None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:46.419864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e0dd4-6417-4dd7-bb36-009acaac50a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:46.420465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e0dd4-6417-4dd7-bb36-009acaac50a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:46.420734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e0dd4-6417-4dd7-bb36-009acaac50a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:46.421008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e0dd4-6417-4dd7-bb36-009acaac50a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:46.429333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e0dd4-6417-4dd7-bb36-009acaac50a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:46.430932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e0dd4-6417-4dd7-bb36-009acaac50a5 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:46.439931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1182/2389] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:46.533507 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cc4e1-8476-4f31-8ae1-d10ab5ecd6d8 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:46.533791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cc4e1-8476-4f31-8ae1-d10ab5ecd6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:46.534607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cc4e1-8476-4f31-8ae1-d10ab5ecd6d8 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:46.536216 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1208/2390] 10.222.0.70 () {54 vars in 921 bytes} [Wed Aug 24 16:55:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:55:46.542286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2e7cd-c668-4d2b-9a3a-2e52f9851455 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:46.542966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2e7cd-c668-4d2b-9a3a-2e52f9851455 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:46.542966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2e7cd-c668-4d2b-9a3a-2e52f9851455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:46.562984 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-fdd2e7cd-c668-4d2b-9a3a-2e52f9851455 None None] MFA Rules not processed for user `e333fca7790244f6a4ebceabb55faba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:46.564335 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-fdd2e7cd-c668-4d2b-9a3a-2e52f9851455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:46.574361 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1183/2391] 10.222.0.70 () {58 vars in 1018 bytes} [Wed Aug 24 16:55:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:46.588541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230cc4e1-8476-4f31-8ae1-d10ab5ecd6d8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:46.600001 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230cc4e1-8476-4f31-8ae1-d10ab5ecd6d8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:46.604131 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad22b9-8dcf-4c21-81b2-0c352f86235c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:46.604766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad22b9-8dcf-4c21-81b2-0c352f86235c service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:46.605045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad22b9-8dcf-4c21-81b2-0c352f86235c service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:46.605316 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad22b9-8dcf-4c21-81b2-0c352f86235c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:46.615751 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad22b9-8dcf-4c21-81b2-0c352f86235c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:46.617247 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad22b9-8dcf-4c21-81b2-0c352f86235c service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:46.628054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1209/2392] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:49.159038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2e7cd-c668-4d2b-9a3a-2e52f9851455 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:49.160726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-fdd2e7cd-c668-4d2b-9a3a-2e52f9851455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:49.176969 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2e7cd-c668-4d2b-9a3a-2e52f9851455 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:49.183095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b78f8-5ea8-4e36-a788-d795a1ec98f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:49.183766 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b78f8-5ea8-4e36-a788-d795a1ec98f8 service placement] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:49.184044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b78f8-5ea8-4e36-a788-d795a1ec98f8 service placement] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:49.184469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b78f8-5ea8-4e36-a788-d795a1ec98f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:49.186184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-441b78f8-5ea8-4e36-a788-d795a1ec98f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:49.197920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b78f8-5ea8-4e36-a788-d795a1ec98f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:49.200389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b78f8-5ea8-4e36-a788-d795a1ec98f8 service placement] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:49.209345 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1184/2393] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:49.300128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad22b9-8dcf-4c21-81b2-0c352f86235c service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:49.301660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0fad22b9-8dcf-4c21-81b2-0c352f86235c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:49.325376 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad22b9-8dcf-4c21-81b2-0c352f86235c service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:49.330138 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5da8e-4cae-448c-a902-791d31a3cee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:49.330776 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5da8e-4cae-448c-a902-791d31a3cee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:49.331057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5da8e-4cae-448c-a902-791d31a3cee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:49.331336 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5da8e-4cae-448c-a902-791d31a3cee6 admin admin] PATH_INFO: `/v3/users/45913fc3d9ba42ea9e971db25e656c61` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:49.332526 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5da8e-4cae-448c-a902-791d31a3cee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45913fc3d9ba42ea9e971db25e656c61)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:49.334490 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5da8e-4cae-448c-a902-791d31a3cee6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:49.531162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-32e5da8e-4cae-448c-a902-791d31a3cee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45913fc3d9ba42ea9e971db25e656c61'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:49.531933 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-32e5da8e-4cae-448c-a902-791d31a3cee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45913fc3d9ba42ea9e971db25e656c61'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:49.532670 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-32e5da8e-4cae-448c-a902-791d31a3cee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45913fc3d9ba42ea9e971db25e656c61'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:49.596250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-32e5da8e-4cae-448c-a902-791d31a3cee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45913fc3d9ba42ea9e971db25e656c61'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:49.604798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-32e5da8e-4cae-448c-a902-791d31a3cee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45913fc3d9ba42ea9e971db25e656c61'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:49.623338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1210/2394] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:55:49 2022] DELETE /identity/v3/users/45913fc3d9ba42ea9e971db25e656c61 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:49.868564 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b78f8-5ea8-4e36-a788-d795a1ec98f8 service placement] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:49.885891 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b78f8-5ea8-4e36-a788-d795a1ec98f8 service placement] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:49.973513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863141d0-4368-48da-b0fa-fc09cb43a472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:49.974963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863141d0-4368-48da-b0fa-fc09cb43a472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:49.975418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863141d0-4368-48da-b0fa-fc09cb43a472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:49.975923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863141d0-4368-48da-b0fa-fc09cb43a472 admin admin] PATH_INFO: `/v3/projects/42f5635b43e74a96a56d56cccbc111b5` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:49.977394 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863141d0-4368-48da-b0fa-fc09cb43a472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42f5635b43e74a96a56d56cccbc111b5)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:49.979495 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863141d0-4368-48da-b0fa-fc09cb43a472 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:50.130001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-863141d0-4368-48da-b0fa-fc09cb43a472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42f5635b43e74a96a56d56cccbc111b5', 'request_id': 'req-863141d0-4368-48da-b0fa-fc09cb43a472'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:50.136871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-863141d0-4368-48da-b0fa-fc09cb43a472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42f5635b43e74a96a56d56cccbc111b5', 'request_id': 'req-863141d0-4368-48da-b0fa-fc09cb43a472'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:50.142240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-863141d0-4368-48da-b0fa-fc09cb43a472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42f5635b43e74a96a56d56cccbc111b5', 'request_id': 'req-863141d0-4368-48da-b0fa-fc09cb43a472'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:50.230000 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-863141d0-4368-48da-b0fa-fc09cb43a472 admin admin] The token cache is being invalidate because project 42f5635b43e74a96a56d56cccbc111b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:55:50.230688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-863141d0-4368-48da-b0fa-fc09cb43a472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:50.230985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-863141d0-4368-48da-b0fa-fc09cb43a472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:50.232600 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1185/2395] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:55:49 2022] DELETE /identity/v3/projects/42f5635b43e74a96a56d56cccbc111b5 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:50.237543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5188d3c-f843-49d9-95f8-f6d143274574 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:50.237832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5188d3c-f843-49d9-95f8-f6d143274574 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:50.238106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5188d3c-f843-49d9-95f8-f6d143274574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:50.263413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c5188d3c-f843-49d9-95f8-f6d143274574 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:50.263413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c5188d3c-f843-49d9-95f8-f6d143274574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:50.299971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:50.317120 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1211/2396] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:55:50.325372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d146dd1-f31f-4ed7-bee0-ccc56187e5c5 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:50.325665 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d146dd1-f31f-4ed7-bee0-ccc56187e5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:50.325967 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d146dd1-f31f-4ed7-bee0-ccc56187e5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:50.345400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-4d146dd1-f31f-4ed7-bee0-ccc56187e5c5 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:50.346598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-4d146dd1-f31f-4ed7-bee0-ccc56187e5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:50.354625 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1186/2397] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:50.358806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5188d3c-f843-49d9-95f8-f6d143274574 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:50.368154 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5188d3c-f843-49d9-95f8-f6d143274574 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:50.373224 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764f55d-61cc-4118-9f49-7218edb71d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:50.373823 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764f55d-61cc-4118-9f49-7218edb71d6b admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:50.374108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764f55d-61cc-4118-9f49-7218edb71d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:50.374383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764f55d-61cc-4118-9f49-7218edb71d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:50.375036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764f55d-61cc-4118-9f49-7218edb71d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:55:50.375343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764f55d-61cc-4118-9f49-7218edb71d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:50.376979 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764f55d-61cc-4118-9f49-7218edb71d6b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:50.390205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1212/2398] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:55:50.393948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d146dd1-f31f-4ed7-bee0-ccc56187e5c5 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:50.403911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d146dd1-f31f-4ed7-bee0-ccc56187e5c5 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:50.408643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43914236-8bb3-484d-bcd7-081ac2513d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:50.409239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43914236-8bb3-484d-bcd7-081ac2513d96 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:50.409524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43914236-8bb3-484d-bcd7-081ac2513d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:50.409794 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43914236-8bb3-484d-bcd7-081ac2513d96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:50.410419 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43914236-8bb3-484d-bcd7-081ac2513d96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:50.412320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43914236-8bb3-484d-bcd7-081ac2513d96 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:50.513883 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1187/2399] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:55:50 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:55:50.517909 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764f55d-61cc-4118-9f49-7218edb71d6b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:50.528593 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764f55d-61cc-4118-9f49-7218edb71d6b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:50.584870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993f66c-b5da-4796-9986-5ce26ff48cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:50.585477 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993f66c-b5da-4796-9986-5ce26ff48cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:50.585759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993f66c-b5da-4796-9986-5ce26ff48cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:50.586034 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993f66c-b5da-4796-9986-5ce26ff48cc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:50.586657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6993f66c-b5da-4796-9986-5ce26ff48cc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:50.589233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6993f66c-b5da-4796-9986-5ce26ff48cc2 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:50.719213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1213/2400] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:55:50 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:55:50.723206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43914236-8bb3-484d-bcd7-081ac2513d96 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:50.733430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43914236-8bb3-484d-bcd7-081ac2513d96 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:50.737937 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ac881-70f4-46af-823c-02fabaca320d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:50.738531 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ac881-70f4-46af-823c-02fabaca320d admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:50.738812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ac881-70f4-46af-823c-02fabaca320d admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:50.739085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ac881-70f4-46af-823c-02fabaca320d admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:50.739720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ac881-70f4-46af-823c-02fabaca320d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:50.741354 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ac881-70f4-46af-823c-02fabaca320d admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:50.751155 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1188/2401] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:55:50.755207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993f66c-b5da-4796-9986-5ce26ff48cc2 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:50.765639 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993f66c-b5da-4796-9986-5ce26ff48cc2 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:50.770117 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713baa0b-0602-4b3a-94a8-c509271d8e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:50.770729 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713baa0b-0602-4b3a-94a8-c509271d8e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:50.771008 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713baa0b-0602-4b3a-94a8-c509271d8e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:50.771277 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713baa0b-0602-4b3a-94a8-c509271d8e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:50.771909 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713baa0b-0602-4b3a-94a8-c509271d8e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:50.773536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713baa0b-0602-4b3a-94a8-c509271d8e73 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:50.784854 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1214/2402] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:55:50.788880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ac881-70f4-46af-823c-02fabaca320d admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:50.798857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ac881-70f4-46af-823c-02fabaca320d admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:50.803672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9efcbc-bacc-47d2-8156-a1f99c86da29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:50.804321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9efcbc-bacc-47d2-8156-a1f99c86da29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:50.804634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9efcbc-bacc-47d2-8156-a1f99c86da29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:50.804910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9efcbc-bacc-47d2-8156-a1f99c86da29 admin admin] PATH_INFO: `/v3/projects/26bd4ee5713f49b7af4b823f270e7b5f/users/41b5b802c50d4d1cbb6314eebed4c75d/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:50.819325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9efcbc-bacc-47d2-8156-a1f99c86da29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26bd4ee5713f49b7af4b823f270e7b5f, user_id=41b5b802c50d4d1cbb6314eebed4c75d, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:50.822551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9efcbc-bacc-47d2-8156-a1f99c86da29 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:50.824746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:50.911769 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1189/2403] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:55:50 2022] PUT /identity/v3/projects/26bd4ee5713f49b7af4b823f270e7b5f/users/41b5b802c50d4d1cbb6314eebed4c75d/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:50.922874 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed4bda-5469-4bdf-873b-61f7b47f0553 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:50.923168 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed4bda-5469-4bdf-873b-61f7b47f0553 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:50.923467 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed4bda-5469-4bdf-873b-61f7b47f0553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:50.952016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-1bed4bda-5469-4bdf-873b-61f7b47f0553 None None] MFA Rules not processed for user `41b5b802c50d4d1cbb6314eebed4c75d`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:50.953281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-1bed4bda-5469-4bdf-873b-61f7b47f0553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:51.018782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1215/2404] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:55:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:51.029455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9efcbc-bacc-47d2-8156-a1f99c86da29 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:51.031179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2c9efcbc-bacc-47d2-8156-a1f99c86da29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:51.106106 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9efcbc-bacc-47d2-8156-a1f99c86da29 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:51.111669 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e29a89-2cbe-4f40-9627-64fb9be97e83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:51.112279 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e29a89-2cbe-4f40-9627-64fb9be97e83 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:51.112591 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e29a89-2cbe-4f40-9627-64fb9be97e83 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:51.112865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e29a89-2cbe-4f40-9627-64fb9be97e83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:51.121184 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e29a89-2cbe-4f40-9627-64fb9be97e83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:51.122755 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e29a89-2cbe-4f40-9627-64fb9be97e83 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:51.131944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1190/2405] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:56:01.013861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:01.127587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:INSERT=1,keystone:SELECT=36 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:01.749311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed4bda-5469-4bdf-873b-61f7b47f0553 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:01.864993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed4bda-5469-4bdf-873b-61f7b47f0553 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:01.870111 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f01225-57de-42f2-b0de-fb3d8bf74382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:01.870742 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f01225-57de-42f2-b0de-fb3d8bf74382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:01.871014 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f01225-57de-42f2-b0de-fb3d8bf74382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:01.871283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f01225-57de-42f2-b0de-fb3d8bf74382 admin admin] PATH_INFO: `/v3/users/41b5b802c50d4d1cbb6314eebed4c75d` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:01.872430 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f01225-57de-42f2-b0de-fb3d8bf74382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41b5b802c50d4d1cbb6314eebed4c75d)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:01.874340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f01225-57de-42f2-b0de-fb3d8bf74382 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:02.086264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c2f01225-57de-42f2-b0de-fb3d8bf74382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41b5b802c50d4d1cbb6314eebed4c75d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:02.086919 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c2f01225-57de-42f2-b0de-fb3d8bf74382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41b5b802c50d4d1cbb6314eebed4c75d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:02.088122 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c2f01225-57de-42f2-b0de-fb3d8bf74382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41b5b802c50d4d1cbb6314eebed4c75d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:02.186188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c2f01225-57de-42f2-b0de-fb3d8bf74382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41b5b802c50d4d1cbb6314eebed4c75d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:02.193511 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c2f01225-57de-42f2-b0de-fb3d8bf74382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41b5b802c50d4d1cbb6314eebed4c75d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:02.210369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1216/2406] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:56:01 2022] DELETE /identity/v3/users/41b5b802c50d4d1cbb6314eebed4c75d => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:02.215331 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d86d6-f1b2-4fb3-9c85-cabe378effbd None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:02.215798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d86d6-f1b2-4fb3-9c85-cabe378effbd None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:02.215921 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d86d6-f1b2-4fb3-9c85-cabe378effbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:02.264446 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-d34d86d6-f1b2-4fb3-9c85-cabe378effbd None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:02.265651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d34d86d6-f1b2-4fb3-9c85-cabe378effbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:02.323222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1191/2407] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:02.332912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f01225-57de-42f2-b0de-fb3d8bf74382 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:02.334298 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c2f01225-57de-42f2-b0de-fb3d8bf74382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:02.376455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:02.408191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f01225-57de-42f2-b0de-fb3d8bf74382 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:02.411795 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079c0fd-aa02-406a-9275-0f594c7deac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:02.412417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079c0fd-aa02-406a-9275-0f594c7deac4 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:02.412717 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079c0fd-aa02-406a-9275-0f594c7deac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:02.412994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079c0fd-aa02-406a-9275-0f594c7deac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:02.422355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079c0fd-aa02-406a-9275-0f594c7deac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:02.428387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079c0fd-aa02-406a-9275-0f594c7deac4 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:02.433613 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1217/2408] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:56:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:56:02.888021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d86d6-f1b2-4fb3-9c85-cabe378effbd None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:02.888021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d34d86d6-f1b2-4fb3-9c85-cabe378effbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:02.900265 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d86d6-f1b2-4fb3-9c85-cabe378effbd None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:02.905039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93afa71-d016-4451-bfd4-6ec4245b6928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:02.905707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93afa71-d016-4451-bfd4-6ec4245b6928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:02.905972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93afa71-d016-4451-bfd4-6ec4245b6928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:02.906236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93afa71-d016-4451-bfd4-6ec4245b6928 admin admin] PATH_INFO: `/v3/projects/26bd4ee5713f49b7af4b823f270e7b5f` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:02.907406 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93afa71-d016-4451-bfd4-6ec4245b6928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26bd4ee5713f49b7af4b823f270e7b5f)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:02.909407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93afa71-d016-4451-bfd4-6ec4245b6928 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:03.055359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c93afa71-d016-4451-bfd4-6ec4245b6928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26bd4ee5713f49b7af4b823f270e7b5f', 'request_id': 'req-c93afa71-d016-4451-bfd4-6ec4245b6928'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:03.061949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c93afa71-d016-4451-bfd4-6ec4245b6928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26bd4ee5713f49b7af4b823f270e7b5f', 'request_id': 'req-c93afa71-d016-4451-bfd4-6ec4245b6928'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:03.067500 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c93afa71-d016-4451-bfd4-6ec4245b6928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26bd4ee5713f49b7af4b823f270e7b5f', 'request_id': 'req-c93afa71-d016-4451-bfd4-6ec4245b6928'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:03.163898 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c93afa71-d016-4451-bfd4-6ec4245b6928 admin admin] The token cache is being invalidate because project 26bd4ee5713f49b7af4b823f270e7b5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:56:03.164194 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c93afa71-d016-4451-bfd4-6ec4245b6928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:03.164880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c93afa71-d016-4451-bfd4-6ec4245b6928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:03.166423 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1192/2409] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:56:02 2022] DELETE /identity/v3/projects/26bd4ee5713f49b7af4b823f270e7b5f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:03.173897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d5bf9-03f7-4b7c-a286-e6d5ec1023b6 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:03.174441 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d5bf9-03f7-4b7c-a286-e6d5ec1023b6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:03.174441 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d5bf9-03f7-4b7c-a286-e6d5ec1023b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:03.236725 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0a2d5bf9-03f7-4b7c-a286-e6d5ec1023b6 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:03.238050 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0a2d5bf9-03f7-4b7c-a286-e6d5ec1023b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:03.301071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1218/2410] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:56:03.309512 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77872185-ce4a-4548-9a3d-79d57b203e2b None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:03.310016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77872185-ce4a-4548-9a3d-79d57b203e2b None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:03.310920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77872185-ce4a-4548-9a3d-79d57b203e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:03.332372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-77872185-ce4a-4548-9a3d-79d57b203e2b None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:03.333632 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-77872185-ce4a-4548-9a3d-79d57b203e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:03.341646 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1193/2411] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:03.345930 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d5bf9-03f7-4b7c-a286-e6d5ec1023b6 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:03.356044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d5bf9-03f7-4b7c-a286-e6d5ec1023b6 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:03.361574 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14621733-3d26-45f1-bed4-9bdcc0b5811b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:03.366694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14621733-3d26-45f1-bed4-9bdcc0b5811b admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:03.366694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14621733-3d26-45f1-bed4-9bdcc0b5811b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:03.366694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14621733-3d26-45f1-bed4-9bdcc0b5811b admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:03.366694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14621733-3d26-45f1-bed4-9bdcc0b5811b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:56:03.366694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14621733-3d26-45f1-bed4-9bdcc0b5811b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:03.366694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14621733-3d26-45f1-bed4-9bdcc0b5811b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:03.382203 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1219/2412] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:56:03.383315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77872185-ce4a-4548-9a3d-79d57b203e2b None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:03.392664 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77872185-ce4a-4548-9a3d-79d57b203e2b None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:03.401405 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef528c-c99a-4448-8d4e-d0821ac8ef49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:03.401405 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef528c-c99a-4448-8d4e-d0821ac8ef49 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:03.401405 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef528c-c99a-4448-8d4e-d0821ac8ef49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:03.401698 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef528c-c99a-4448-8d4e-d0821ac8ef49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:03.402111 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef528c-c99a-4448-8d4e-d0821ac8ef49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:03.404251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef528c-c99a-4448-8d4e-d0821ac8ef49 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:03.505132 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1194/2413] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:56:03 2022] POST /identity/v3/projects => generated 355 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:56:03.509339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14621733-3d26-45f1-bed4-9bdcc0b5811b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:03.519399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14621733-3d26-45f1-bed4-9bdcc0b5811b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:03.580617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc9f11-2c77-4d64-87c1-79ab9ea4d250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:03.585270 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc9f11-2c77-4d64-87c1-79ab9ea4d250 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:03.585270 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc9f11-2c77-4d64-87c1-79ab9ea4d250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:03.585270 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc9f11-2c77-4d64-87c1-79ab9ea4d250 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:03.585270 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc9f11-2c77-4d64-87c1-79ab9ea4d250 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:03.585270 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc9f11-2c77-4d64-87c1-79ab9ea4d250 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:03.735666 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1220/2414] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:56:03 2022] POST /identity/v3/users => generated 278 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:56:03.739628 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef528c-c99a-4448-8d4e-d0821ac8ef49 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:03.750767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef528c-c99a-4448-8d4e-d0821ac8ef49 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:03.755632 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a516e-a6b7-4110-a2a8-3157663dc6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:03.756487 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a516e-a6b7-4110-a2a8-3157663dc6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:03.756626 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a516e-a6b7-4110-a2a8-3157663dc6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:03.756840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a516e-a6b7-4110-a2a8-3157663dc6b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:03.757477 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a516e-a6b7-4110-a2a8-3157663dc6b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:03.759711 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a516e-a6b7-4110-a2a8-3157663dc6b6 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:03.769968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1195/2415] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:56:03.773867 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc9f11-2c77-4d64-87c1-79ab9ea4d250 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:03.784003 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc9f11-2c77-4d64-87c1-79ab9ea4d250 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:03.789386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34932038-ee99-418b-b7a3-ba13577b1b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:03.790005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34932038-ee99-418b-b7a3-ba13577b1b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:03.790341 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34932038-ee99-418b-b7a3-ba13577b1b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:03.790617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34932038-ee99-418b-b7a3-ba13577b1b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:03.791269 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34932038-ee99-418b-b7a3-ba13577b1b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:03.792928 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34932038-ee99-418b-b7a3-ba13577b1b05 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:03.809116 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1221/2416] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:56:03.811126 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a516e-a6b7-4110-a2a8-3157663dc6b6 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:03.822430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a516e-a6b7-4110-a2a8-3157663dc6b6 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:03.827579 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37182450-8d58-47ed-894d-bd8d14914bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:03.828174 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37182450-8d58-47ed-894d-bd8d14914bea admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:03.828562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37182450-8d58-47ed-894d-bd8d14914bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:03.828914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37182450-8d58-47ed-894d-bd8d14914bea admin admin] PATH_INFO: `/v3/projects/841fd58b06f34249a2d26b18e9618b3a/users/e628846d164a455a9f46eb8ec1c8609f/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:03.844211 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37182450-8d58-47ed-894d-bd8d14914bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=841fd58b06f34249a2d26b18e9618b3a, user_id=e628846d164a455a9f46eb8ec1c8609f, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:03.847069 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37182450-8d58-47ed-894d-bd8d14914bea admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:03.849443 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:03.941539 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1196/2417] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:56:03 2022] PUT /identity/v3/projects/841fd58b06f34249a2d26b18e9618b3a/users/e628846d164a455a9f46eb8ec1c8609f/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:03.952382 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79477f-e2fa-4b0c-9c8f-54011c049010 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:03.952699 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79477f-e2fa-4b0c-9c8f-54011c049010 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:03.953830 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79477f-e2fa-4b0c-9c8f-54011c049010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:03.965844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:03.980231 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ab79477f-e2fa-4b0c-9c8f-54011c049010 None None] MFA Rules not processed for user `e628846d164a455a9f46eb8ec1c8609f`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:03.981483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ab79477f-e2fa-4b0c-9c8f-54011c049010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:04.045943 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1222/2418] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:04.050580 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37182450-8d58-47ed-894d-bd8d14914bea admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:04.112133 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37182450-8d58-47ed-894d-bd8d14914bea admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:04.117044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c640553-dae9-49bd-ae36-a137509a88f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:04.117653 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c640553-dae9-49bd-ae36-a137509a88f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:04.117933 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c640553-dae9-49bd-ae36-a137509a88f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:04.118290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c640553-dae9-49bd-ae36-a137509a88f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:04.118946 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c640553-dae9-49bd-ae36-a137509a88f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:04.120870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c640553-dae9-49bd-ae36-a137509a88f2 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:04.268513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1197/2419] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:56:04 2022] POST /identity/v3/projects => generated 357 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:56:04.272263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79477f-e2fa-4b0c-9c8f-54011c049010 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:04.283634 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79477f-e2fa-4b0c-9c8f-54011c049010 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:04.340491 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480eb272-5c3c-46ef-a27e-b73486679541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:04.341235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480eb272-5c3c-46ef-a27e-b73486679541 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:04.341513 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480eb272-5c3c-46ef-a27e-b73486679541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:04.341782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480eb272-5c3c-46ef-a27e-b73486679541 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:04.342402 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480eb272-5c3c-46ef-a27e-b73486679541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:04.344388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480eb272-5c3c-46ef-a27e-b73486679541 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:04.458860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1223/2420] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:56:04 2022] POST /identity/v3/users => generated 285 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:56:04.463220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c640553-dae9-49bd-ae36-a137509a88f2 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:04.480006 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c640553-dae9-49bd-ae36-a137509a88f2 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:04.485125 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603375b7-a695-4528-accc-b8fae8f1b443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:04.490197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603375b7-a695-4528-accc-b8fae8f1b443 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:04.490197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603375b7-a695-4528-accc-b8fae8f1b443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:04.490197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603375b7-a695-4528-accc-b8fae8f1b443 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:04.490197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603375b7-a695-4528-accc-b8fae8f1b443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:04.490197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603375b7-a695-4528-accc-b8fae8f1b443 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:04.499563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1198/2421] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:56:04.504198 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480eb272-5c3c-46ef-a27e-b73486679541 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:04.515180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480eb272-5c3c-46ef-a27e-b73486679541 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:04.524455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adbe18-0a85-499d-9c8f-5b0d660cd8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:04.524455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adbe18-0a85-499d-9c8f-5b0d660cd8fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:04.524455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adbe18-0a85-499d-9c8f-5b0d660cd8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:04.524455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adbe18-0a85-499d-9c8f-5b0d660cd8fe admin admin] PATH_INFO: `/v3/projects/2a7127d49dd84424bfb96907e8ac165c/users/d7690aba95b146088baf366616784a36/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:04.535954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adbe18-0a85-499d-9c8f-5b0d660cd8fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7127d49dd84424bfb96907e8ac165c, user_id=d7690aba95b146088baf366616784a36, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:04.538821 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adbe18-0a85-499d-9c8f-5b0d660cd8fe admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:04.638473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1224/2422] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:56:04 2022] PUT /identity/v3/projects/2a7127d49dd84424bfb96907e8ac165c/users/d7690aba95b146088baf366616784a36/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:56:04.642372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603375b7-a695-4528-accc-b8fae8f1b443 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:04.652112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603375b7-a695-4528-accc-b8fae8f1b443 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:04.707084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc26d2-3fa5-44a2-b876-0952b60186b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:04.707687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc26d2-3fa5-44a2-b876-0952b60186b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:04.707997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc26d2-3fa5-44a2-b876-0952b60186b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:04.708271 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc26d2-3fa5-44a2-b876-0952b60186b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:04.708943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc26d2-3fa5-44a2-b876-0952b60186b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:04.710684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc26d2-3fa5-44a2-b876-0952b60186b7 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:04.720110 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1199/2423] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:56:04.724350 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adbe18-0a85-499d-9c8f-5b0d660cd8fe admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:04.735039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adbe18-0a85-499d-9c8f-5b0d660cd8fe admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:04.739857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da071ff-88c7-4ff4-ba9b-caa243a6f47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:04.740588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da071ff-88c7-4ff4-ba9b-caa243a6f47e admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:04.740870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da071ff-88c7-4ff4-ba9b-caa243a6f47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:04.741143 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da071ff-88c7-4ff4-ba9b-caa243a6f47e admin admin] PATH_INFO: `/v3/projects/2a7127d49dd84424bfb96907e8ac165c/users/d7690aba95b146088baf366616784a36/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:04.743042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da071ff-88c7-4ff4-ba9b-caa243a6f47e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7127d49dd84424bfb96907e8ac165c, user_id=d7690aba95b146088baf366616784a36, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:04.746555 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da071ff-88c7-4ff4-ba9b-caa243a6f47e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:04.757044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1225/2424] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:56:04 2022] PUT /identity/v3/projects/2a7127d49dd84424bfb96907e8ac165c/users/d7690aba95b146088baf366616784a36/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:04.773719 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa9666-6979-44c3-b75f-bebbc9cf5405 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:04.774197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa9666-6979-44c3-b75f-bebbc9cf5405 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:04.775691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa9666-6979-44c3-b75f-bebbc9cf5405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:04.813899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-baaa9666-6979-44c3-b75f-bebbc9cf5405 None None] MFA Rules not processed for user `d7690aba95b146088baf366616784a36`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:04.815133 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-baaa9666-6979-44c3-b75f-bebbc9cf5405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:04.866233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:04.887871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1200/2425] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:04 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:04.899070 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da071ff-88c7-4ff4-ba9b-caa243a6f47e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:04.901370 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9da071ff-88c7-4ff4-ba9b-caa243a6f47e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:04.953357 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:04.956974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da071ff-88c7-4ff4-ba9b-caa243a6f47e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:04.960572 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9c534-dc5e-48b6-8509-8bf266ea3e03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:04.961172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9c534-dc5e-48b6-8509-8bf266ea3e03 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:04.961448 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9c534-dc5e-48b6-8509-8bf266ea3e03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:04.961716 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9c534-dc5e-48b6-8509-8bf266ea3e03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:04.971073 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9c534-dc5e-48b6-8509-8bf266ea3e03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:04.972503 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9c534-dc5e-48b6-8509-8bf266ea3e03 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:04.981669 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1226/2426] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:56:04 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:56:05.191966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa9666-6979-44c3-b75f-bebbc9cf5405 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:05.202101 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa9666-6979-44c3-b75f-bebbc9cf5405 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:05.212534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d09525-6831-4a30-879b-9c900afc0dab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:05.212534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d09525-6831-4a30-879b-9c900afc0dab service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:05.212534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d09525-6831-4a30-879b-9c900afc0dab service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:05.212534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d09525-6831-4a30-879b-9c900afc0dab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:05.264879 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d09525-6831-4a30-879b-9c900afc0dab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:05.266369 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d09525-6831-4a30-879b-9c900afc0dab service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:05.276972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1201/2427] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:56:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:56:07.184021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9c534-dc5e-48b6-8509-8bf266ea3e03 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:07.185439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dcb9c534-dc5e-48b6-8509-8bf266ea3e03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:07.286683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9c534-dc5e-48b6-8509-8bf266ea3e03 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:07.292186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf0509-efcc-4f19-95b4-efb3279b7dc2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:07.292823 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf0509-efcc-4f19-95b4-efb3279b7dc2 service placement] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:07.293108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf0509-efcc-4f19-95b4-efb3279b7dc2 service placement] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:07.293525 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf0509-efcc-4f19-95b4-efb3279b7dc2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:07.295242 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bdcf0509-efcc-4f19-95b4-efb3279b7dc2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:07.306753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf0509-efcc-4f19-95b4-efb3279b7dc2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:07.308999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf0509-efcc-4f19-95b4-efb3279b7dc2 service placement] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:07.318912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1227/2428] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:56:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:56:15.272976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=33 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:17.314670 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=45 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:24.188247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d09525-6831-4a30-879b-9c900afc0dab service cinder] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:24.250725 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d09525-6831-4a30-879b-9c900afc0dab service cinder] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:24.255525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791c126-430d-47fe-a9b5-b55493146b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:24.256144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791c126-430d-47fe-a9b5-b55493146b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:24.256449 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791c126-430d-47fe-a9b5-b55493146b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:24.256728 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791c126-430d-47fe-a9b5-b55493146b14 admin admin] PATH_INFO: `/v3/users/e628846d164a455a9f46eb8ec1c8609f` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:24.257834 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791c126-430d-47fe-a9b5-b55493146b14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e628846d164a455a9f46eb8ec1c8609f)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:24.259764 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791c126-430d-47fe-a9b5-b55493146b14 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:24.433552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2791c126-430d-47fe-a9b5-b55493146b14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e628846d164a455a9f46eb8ec1c8609f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:24.434201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2791c126-430d-47fe-a9b5-b55493146b14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e628846d164a455a9f46eb8ec1c8609f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:24.434804 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2791c126-430d-47fe-a9b5-b55493146b14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e628846d164a455a9f46eb8ec1c8609f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:24.442362 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2791c126-430d-47fe-a9b5-b55493146b14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e628846d164a455a9f46eb8ec1c8609f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:24.506390 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2791c126-430d-47fe-a9b5-b55493146b14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e628846d164a455a9f46eb8ec1c8609f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:24.526116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1202/2429] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:56:24 2022] DELETE /identity/v3/users/e628846d164a455a9f46eb8ec1c8609f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:24.531639 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1f421-4485-48d4-884a-d4ae4867092c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:24.532208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1f421-4485-48d4-884a-d4ae4867092c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:24.532208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1f421-4485-48d4-884a-d4ae4867092c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:24.553276 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e1a1f421-4485-48d4-884a-d4ae4867092c None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:24.554505 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e1a1f421-4485-48d4-884a-d4ae4867092c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:24.617295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1228/2430] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:24.627621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791c126-430d-47fe-a9b5-b55493146b14 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:24.629386 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2791c126-430d-47fe-a9b5-b55493146b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:24.706406 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791c126-430d-47fe-a9b5-b55493146b14 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:24.709932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a761fe-9335-4315-bd7e-75892fd6334f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:24.710529 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a761fe-9335-4315-bd7e-75892fd6334f service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:24.710808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a761fe-9335-4315-bd7e-75892fd6334f service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:24.711080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a761fe-9335-4315-bd7e-75892fd6334f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:24.719798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a761fe-9335-4315-bd7e-75892fd6334f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:24.721240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a761fe-9335-4315-bd7e-75892fd6334f service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:24.730676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1203/2431] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:56:25.123255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1f421-4485-48d4-884a-d4ae4867092c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:25.124737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e1a1f421-4485-48d4-884a-d4ae4867092c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:25.137709 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1f421-4485-48d4-884a-d4ae4867092c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:25.142281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299cef5-fbf0-4efc-ac57-550bb223de0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:25.142911 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299cef5-fbf0-4efc-ac57-550bb223de0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:25.143184 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299cef5-fbf0-4efc-ac57-550bb223de0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:25.143488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299cef5-fbf0-4efc-ac57-550bb223de0d admin admin] PATH_INFO: `/v3/projects/841fd58b06f34249a2d26b18e9618b3a` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:25.144619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299cef5-fbf0-4efc-ac57-550bb223de0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=841fd58b06f34249a2d26b18e9618b3a)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:25.146514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299cef5-fbf0-4efc-ac57-550bb223de0d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:25.241512 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a299cef5-fbf0-4efc-ac57-550bb223de0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '841fd58b06f34249a2d26b18e9618b3a', 'request_id': 'req-a299cef5-fbf0-4efc-ac57-550bb223de0d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:25.247770 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a299cef5-fbf0-4efc-ac57-550bb223de0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '841fd58b06f34249a2d26b18e9618b3a', 'request_id': 'req-a299cef5-fbf0-4efc-ac57-550bb223de0d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:25.253328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a299cef5-fbf0-4efc-ac57-550bb223de0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '841fd58b06f34249a2d26b18e9618b3a', 'request_id': 'req-a299cef5-fbf0-4efc-ac57-550bb223de0d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:25.316358 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a299cef5-fbf0-4efc-ac57-550bb223de0d admin admin] The token cache is being invalidate because project 841fd58b06f34249a2d26b18e9618b3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:56:25.316681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a299cef5-fbf0-4efc-ac57-550bb223de0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:25.317294 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a299cef5-fbf0-4efc-ac57-550bb223de0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:25.318776 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1229/2432] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:56:25 2022] DELETE /identity/v3/projects/841fd58b06f34249a2d26b18e9618b3a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:25.322858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a761fe-9335-4315-bd7e-75892fd6334f service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:25.328384 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:25.333750 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a761fe-9335-4315-bd7e-75892fd6334f service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:25.389599 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ce396-3465-4a26-9473-f873c156bfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:25.390303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ce396-3465-4a26-9473-f873c156bfc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:25.390583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ce396-3465-4a26-9473-f873c156bfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:25.390875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ce396-3465-4a26-9473-f873c156bfc7 admin admin] PATH_INFO: `/v3/users/d7690aba95b146088baf366616784a36` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:25.392039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ce396-3465-4a26-9473-f873c156bfc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7690aba95b146088baf366616784a36)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:25.394036 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ce396-3465-4a26-9473-f873c156bfc7 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:25.594120 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-747ce396-3465-4a26-9473-f873c156bfc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7690aba95b146088baf366616784a36'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:25.594787 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-747ce396-3465-4a26-9473-f873c156bfc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7690aba95b146088baf366616784a36'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:25.595400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-747ce396-3465-4a26-9473-f873c156bfc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7690aba95b146088baf366616784a36'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:25.603141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-747ce396-3465-4a26-9473-f873c156bfc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7690aba95b146088baf366616784a36'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:25.664742 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-747ce396-3465-4a26-9473-f873c156bfc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7690aba95b146088baf366616784a36'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:25.683129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1204/2433] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:56:25 2022] DELETE /identity/v3/users/d7690aba95b146088baf366616784a36 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:26.114081 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299cef5-fbf0-4efc-ac57-550bb223de0d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:26.115586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a299cef5-fbf0-4efc-ac57-550bb223de0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:26.128840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299cef5-fbf0-4efc-ac57-550bb223de0d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:26.185694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49be2a-2330-4130-959d-dbb7feb73281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:26.186332 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49be2a-2330-4130-959d-dbb7feb73281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:26.186615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49be2a-2330-4130-959d-dbb7feb73281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:26.186897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49be2a-2330-4130-959d-dbb7feb73281 admin admin] PATH_INFO: `/v3/projects/2a7127d49dd84424bfb96907e8ac165c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:26.188014 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49be2a-2330-4130-959d-dbb7feb73281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a7127d49dd84424bfb96907e8ac165c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:26.189975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49be2a-2330-4130-959d-dbb7feb73281 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:26.296109 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:26.311989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6c49be2a-2330-4130-959d-dbb7feb73281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a7127d49dd84424bfb96907e8ac165c', 'request_id': 'req-6c49be2a-2330-4130-959d-dbb7feb73281'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:26.318147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6c49be2a-2330-4130-959d-dbb7feb73281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a7127d49dd84424bfb96907e8ac165c', 'request_id': 'req-6c49be2a-2330-4130-959d-dbb7feb73281'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:26.323708 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6c49be2a-2330-4130-959d-dbb7feb73281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a7127d49dd84424bfb96907e8ac165c', 'request_id': 'req-6c49be2a-2330-4130-959d-dbb7feb73281'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:26.380629 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6c49be2a-2330-4130-959d-dbb7feb73281 admin admin] The token cache is being invalidate because project 2a7127d49dd84424bfb96907e8ac165c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:56:26.380921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6c49be2a-2330-4130-959d-dbb7feb73281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:26.381631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6c49be2a-2330-4130-959d-dbb7feb73281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:26.383170 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1230/2434] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:56:26 2022] DELETE /identity/v3/projects/2a7127d49dd84424bfb96907e8ac165c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:26.389743 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973c7ed-a7bb-4358-bc56-004048349bfd None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:26.390203 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973c7ed-a7bb-4358-bc56-004048349bfd None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:26.390300 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973c7ed-a7bb-4358-bc56-004048349bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:26.410083 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-a973c7ed-a7bb-4358-bc56-004048349bfd None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:26.411287 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a973c7ed-a7bb-4358-bc56-004048349bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:26.468887 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1205/2435] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:26.476776 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cfa93-971a-4d4b-bfd3-3bca287f4673 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:26.477083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cfa93-971a-4d4b-bfd3-3bca287f4673 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:26.477361 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cfa93-971a-4d4b-bfd3-3bca287f4673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:26.496747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e38cfa93-971a-4d4b-bfd3-3bca287f4673 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:26.497963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e38cfa93-971a-4d4b-bfd3-3bca287f4673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:26.506127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1231/2436] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:56:26.510515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973c7ed-a7bb-4358-bc56-004048349bfd None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:26.520844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973c7ed-a7bb-4358-bc56-004048349bfd None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:26.525536 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0004a65-b7a8-4b79-932b-dec4a1a759be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:26.526138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0004a65-b7a8-4b79-932b-dec4a1a759be admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:26.526415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0004a65-b7a8-4b79-932b-dec4a1a759be admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:26.526686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0004a65-b7a8-4b79-932b-dec4a1a759be admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:26.527331 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0004a65-b7a8-4b79-932b-dec4a1a759be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:56:26.527630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0004a65-b7a8-4b79-932b-dec4a1a759be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:26.529899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0004a65-b7a8-4b79-932b-dec4a1a759be admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:26.543384 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1206/2437] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:56:26.546616 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cfa93-971a-4d4b-bfd3-3bca287f4673 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:26.555730 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cfa93-971a-4d4b-bfd3-3bca287f4673 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:26.560667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12d158-7b68-4308-86e4-26865dfefc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:26.561279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12d158-7b68-4308-86e4-26865dfefc1b admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:26.561564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12d158-7b68-4308-86e4-26865dfefc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:26.561833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12d158-7b68-4308-86e4-26865dfefc1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:26.562468 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12d158-7b68-4308-86e4-26865dfefc1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:26.564455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12d158-7b68-4308-86e4-26865dfefc1b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:26.656780 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1232/2438] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:56:26 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:56:26.660649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0004a65-b7a8-4b79-932b-dec4a1a759be admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:26.670744 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0004a65-b7a8-4b79-932b-dec4a1a759be admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:26.675723 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:26.727362 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6fb984-16ee-48be-b410-d3e50bdc86ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:26.727956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fb984-16ee-48be-b410-d3e50bdc86ca admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:26.728233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fb984-16ee-48be-b410-d3e50bdc86ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:26.728537 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fb984-16ee-48be-b410-d3e50bdc86ca admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:26.729156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6fb984-16ee-48be-b410-d3e50bdc86ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:26.731072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6fb984-16ee-48be-b410-d3e50bdc86ca admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:26.823943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1207/2439] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:56:26 2022] POST /identity/v3/users => generated 283 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:56:26.828951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12d158-7b68-4308-86e4-26865dfefc1b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:26.839251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12d158-7b68-4308-86e4-26865dfefc1b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:26.843918 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375b697-a898-41aa-be56-ca642f5c76fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:26.844562 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375b697-a898-41aa-be56-ca642f5c76fd admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:26.844847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375b697-a898-41aa-be56-ca642f5c76fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:26.845124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375b697-a898-41aa-be56-ca642f5c76fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:26.845764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375b697-a898-41aa-be56-ca642f5c76fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:26.847374 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375b697-a898-41aa-be56-ca642f5c76fd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:26.857172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1233/2440] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:56:26.861163 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6fb984-16ee-48be-b410-d3e50bdc86ca admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:26.871047 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6fb984-16ee-48be-b410-d3e50bdc86ca admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:26.876180 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c46ac0-1e62-422d-8431-629c7f3c4bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:26.876819 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c46ac0-1e62-422d-8431-629c7f3c4bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:26.877102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c46ac0-1e62-422d-8431-629c7f3c4bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:26.877377 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c46ac0-1e62-422d-8431-629c7f3c4bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:26.878013 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c46ac0-1e62-422d-8431-629c7f3c4bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:26.879870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c46ac0-1e62-422d-8431-629c7f3c4bd9 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:26.890981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1208/2441] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:56:26.895054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375b697-a898-41aa-be56-ca642f5c76fd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:26.904976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375b697-a898-41aa-be56-ca642f5c76fd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:26.909655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa0b4a-ff30-490d-bfef-bfbcc4ee2120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:26.910237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa0b4a-ff30-490d-bfef-bfbcc4ee2120 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:26.910521 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa0b4a-ff30-490d-bfef-bfbcc4ee2120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:26.910800 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa0b4a-ff30-490d-bfef-bfbcc4ee2120 admin admin] PATH_INFO: `/v3/projects/9566afa9d1e449e6a7c0c613a5641b53/users/709301a39f78489e84c6607c223ba4c8/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:26.925025 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa0b4a-ff30-490d-bfef-bfbcc4ee2120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9566afa9d1e449e6a7c0c613a5641b53, user_id=709301a39f78489e84c6607c223ba4c8, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:26.927672 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa0b4a-ff30-490d-bfef-bfbcc4ee2120 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:27.009270 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1234/2442] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:56:26 2022] PUT /identity/v3/projects/9566afa9d1e449e6a7c0c613a5641b53/users/709301a39f78489e84c6607c223ba4c8/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:27.021564 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b6ab6-cd74-4041-9469-d389083377ec None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:27.022523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b6ab6-cd74-4041-9469-d389083377ec None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:27.022523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b6ab6-cd74-4041-9469-d389083377ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:27.048923 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-d24b6ab6-cd74-4041-9469-d389083377ec None None] MFA Rules not processed for user `709301a39f78489e84c6607c223ba4c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:27.056692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d24b6ab6-cd74-4041-9469-d389083377ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:27.113920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1209/2443] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:27.118468 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa0b4a-ff30-490d-bfef-bfbcc4ee2120 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:27.179165 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa0b4a-ff30-490d-bfef-bfbcc4ee2120 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:27.189816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b25feb-be48-44fc-9265-e7c706f778a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:27.190467 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b25feb-be48-44fc-9265-e7c706f778a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:27.190744 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b25feb-be48-44fc-9265-e7c706f778a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:27.191019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b25feb-be48-44fc-9265-e7c706f778a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:27.191659 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b25feb-be48-44fc-9265-e7c706f778a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:27.193590 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b25feb-be48-44fc-9265-e7c706f778a3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:27.317393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1235/2444] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:56:27 2022] POST /identity/v3/projects => generated 363 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:56:27.321387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b6ab6-cd74-4041-9469-d389083377ec None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:27.331902 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b6ab6-cd74-4041-9469-d389083377ec None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:27.386987 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea3ec9-705c-4407-b049-61111660433a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:27.387591 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea3ec9-705c-4407-b049-61111660433a admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:27.395324 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea3ec9-705c-4407-b049-61111660433a admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:27.395324 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea3ec9-705c-4407-b049-61111660433a admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:27.395324 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea3ec9-705c-4407-b049-61111660433a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:27.395324 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea3ec9-705c-4407-b049-61111660433a admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:27.523751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1210/2445] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:56:27 2022] POST /identity/v3/users => generated 288 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:56:27.527740 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b25feb-be48-44fc-9265-e7c706f778a3 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:27.538199 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b25feb-be48-44fc-9265-e7c706f778a3 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:27.542767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cbcc9-b912-438e-871c-7d2091804589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:27.543358 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cbcc9-b912-438e-871c-7d2091804589 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:27.544297 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cbcc9-b912-438e-871c-7d2091804589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:27.544599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cbcc9-b912-438e-871c-7d2091804589 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:27.545243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cbcc9-b912-438e-871c-7d2091804589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:27.546825 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cbcc9-b912-438e-871c-7d2091804589 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:27.557337 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1236/2446] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:56:27.561479 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea3ec9-705c-4407-b049-61111660433a admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:27.571543 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea3ec9-705c-4407-b049-61111660433a admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:27.576105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a4f9f-59ee-4c70-8a8e-57b455fd58ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:27.576781 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a4f9f-59ee-4c70-8a8e-57b455fd58ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:27.577066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a4f9f-59ee-4c70-8a8e-57b455fd58ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:27.577339 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a4f9f-59ee-4c70-8a8e-57b455fd58ab admin admin] PATH_INFO: `/v3/projects/c5c675d8b33245688707ea63f5faca08/users/f0f8abed64664e41b4a4042bf388a466/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:27.579576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:27.593953 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a4f9f-59ee-4c70-8a8e-57b455fd58ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5c675d8b33245688707ea63f5faca08, user_id=f0f8abed64664e41b4a4042bf388a466, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:27.596709 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a4f9f-59ee-4c70-8a8e-57b455fd58ab admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:27.670459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1211/2447] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:56:27 2022] PUT /identity/v3/projects/c5c675d8b33245688707ea63f5faca08/users/f0f8abed64664e41b4a4042bf388a466/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:27.674488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cbcc9-b912-438e-871c-7d2091804589 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:27.684165 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cbcc9-b912-438e-871c-7d2091804589 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:27.703446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:27.743793 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8e9a6-7692-4a13-959c-5e62356bed6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:27.744522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8e9a6-7692-4a13-959c-5e62356bed6d admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:27.744876 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8e9a6-7692-4a13-959c-5e62356bed6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:27.745218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8e9a6-7692-4a13-959c-5e62356bed6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:27.745936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8e9a6-7692-4a13-959c-5e62356bed6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:27.747613 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8e9a6-7692-4a13-959c-5e62356bed6d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:27.761082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1237/2448] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:56:27.762250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a4f9f-59ee-4c70-8a8e-57b455fd58ab admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:27.772995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a4f9f-59ee-4c70-8a8e-57b455fd58ab admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:27.779426 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b833cf1-282a-477e-ac78-1ae8d842bd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:27.780083 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b833cf1-282a-477e-ac78-1ae8d842bd2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:27.780359 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b833cf1-282a-477e-ac78-1ae8d842bd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:27.780687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b833cf1-282a-477e-ac78-1ae8d842bd2b admin admin] PATH_INFO: `/v3/projects/c5c675d8b33245688707ea63f5faca08/users/f0f8abed64664e41b4a4042bf388a466/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:27.783469 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b833cf1-282a-477e-ac78-1ae8d842bd2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5c675d8b33245688707ea63f5faca08, user_id=f0f8abed64664e41b4a4042bf388a466, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:27.786209 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b833cf1-282a-477e-ac78-1ae8d842bd2b admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:27.797351 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1212/2449] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:56:27 2022] PUT /identity/v3/projects/c5c675d8b33245688707ea63f5faca08/users/f0f8abed64664e41b4a4042bf388a466/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:56:27.807315 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce839462-d20b-4b04-b47f-0a46e05fa4a9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:27.807606 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce839462-d20b-4b04-b47f-0a46e05fa4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:27.807890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce839462-d20b-4b04-b47f-0a46e05fa4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:27.835126 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ce839462-d20b-4b04-b47f-0a46e05fa4a9 None None] MFA Rules not processed for user `f0f8abed64664e41b4a4042bf388a466`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:27.836332 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ce839462-d20b-4b04-b47f-0a46e05fa4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:27.903767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1238/2450] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:27.915303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b833cf1-282a-477e-ac78-1ae8d842bd2b admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:27.917020 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-7b833cf1-282a-477e-ac78-1ae8d842bd2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:27.994934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b833cf1-282a-477e-ac78-1ae8d842bd2b admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:27.998453 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866569e-7abc-498f-9036-5d2a44e581eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:27.999049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866569e-7abc-498f-9036-5d2a44e581eb service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:27.999323 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866569e-7abc-498f-9036-5d2a44e581eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:27.999590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866569e-7abc-498f-9036-5d2a44e581eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:28.008641 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866569e-7abc-498f-9036-5d2a44e581eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:28.010054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866569e-7abc-498f-9036-5d2a44e581eb service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:28.019640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1213/2451] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:56:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:56:28.205890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce839462-d20b-4b04-b47f-0a46e05fa4a9 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:28.215358 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce839462-d20b-4b04-b47f-0a46e05fa4a9 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:28.219481 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4143b6e-fd41-4442-94a8-2fdda7b77788 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:28.220071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4143b6e-fd41-4442-94a8-2fdda7b77788 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:28.220350 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4143b6e-fd41-4442-94a8-2fdda7b77788 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:28.220661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4143b6e-fd41-4442-94a8-2fdda7b77788 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:28.276305 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4143b6e-fd41-4442-94a8-2fdda7b77788 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:28.277690 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4143b6e-fd41-4442-94a8-2fdda7b77788 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:28.286417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1239/2452] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:56:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:56:31.829122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866569e-7abc-498f-9036-5d2a44e581eb service cinder] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:31.889884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866569e-7abc-498f-9036-5d2a44e581eb service cinder] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:31.894313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fb0dd-b58d-426b-9315-2fe1aca26e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:31.894922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fb0dd-b58d-426b-9315-2fe1aca26e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:31.895198 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fb0dd-b58d-426b-9315-2fe1aca26e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:31.895466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fb0dd-b58d-426b-9315-2fe1aca26e3a admin admin] PATH_INFO: `/v3/users/709301a39f78489e84c6607c223ba4c8` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:31.896585 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587fb0dd-b58d-426b-9315-2fe1aca26e3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=709301a39f78489e84c6607c223ba4c8)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:31.898485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587fb0dd-b58d-426b-9315-2fe1aca26e3a admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:32.091048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-587fb0dd-b58d-426b-9315-2fe1aca26e3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '709301a39f78489e84c6607c223ba4c8'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:32.091742 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-587fb0dd-b58d-426b-9315-2fe1aca26e3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '709301a39f78489e84c6607c223ba4c8'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:32.092343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-587fb0dd-b58d-426b-9315-2fe1aca26e3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '709301a39f78489e84c6607c223ba4c8'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:32.097702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:32.099642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-587fb0dd-b58d-426b-9315-2fe1aca26e3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '709301a39f78489e84c6607c223ba4c8'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:32.191912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-587fb0dd-b58d-426b-9315-2fe1aca26e3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '709301a39f78489e84c6607c223ba4c8'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:32.210330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1214/2453] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:56:31 2022] DELETE /identity/v3/users/709301a39f78489e84c6607c223ba4c8 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:32.215013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b73fb-b4cd-4044-848f-8e7c25c3cfe5 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:32.215493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b73fb-b4cd-4044-848f-8e7c25c3cfe5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:32.215602 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b73fb-b4cd-4044-848f-8e7c25c3cfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:32.234297 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-809b73fb-b4cd-4044-848f-8e7c25c3cfe5 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:32.235492 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-809b73fb-b4cd-4044-848f-8e7c25c3cfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:32.276151 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:32.294466 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1240/2454] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:32.304281 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fb0dd-b58d-426b-9315-2fe1aca26e3a admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:32.305701 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-587fb0dd-b58d-426b-9315-2fe1aca26e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:32.362531 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fb0dd-b58d-426b-9315-2fe1aca26e3a admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:32.366415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd5934-fc84-49b5-a450-e2aab25c374e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:32.367024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd5934-fc84-49b5-a450-e2aab25c374e service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:32.367307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd5934-fc84-49b5-a450-e2aab25c374e service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:32.367582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd5934-fc84-49b5-a450-e2aab25c374e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:32.377116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd5934-fc84-49b5-a450-e2aab25c374e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:32.378541 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd5934-fc84-49b5-a450-e2aab25c374e service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:32.388123 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1215/2455] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:56:32.814905 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b73fb-b4cd-4044-848f-8e7c25c3cfe5 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:32.816330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-809b73fb-b4cd-4044-848f-8e7c25c3cfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:32.828223 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b73fb-b4cd-4044-848f-8e7c25c3cfe5 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:32.833335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cbb54-60fe-4860-81de-cddd8a2372fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:32.834412 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cbb54-60fe-4860-81de-cddd8a2372fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:32.834694 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cbb54-60fe-4860-81de-cddd8a2372fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:32.834968 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cbb54-60fe-4860-81de-cddd8a2372fc admin admin] PATH_INFO: `/v3/projects/9566afa9d1e449e6a7c0c613a5641b53` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:32.836095 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cbb54-60fe-4860-81de-cddd8a2372fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9566afa9d1e449e6a7c0c613a5641b53)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:32.838036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cbb54-60fe-4860-81de-cddd8a2372fc admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:32.961386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be7cbb54-60fe-4860-81de-cddd8a2372fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9566afa9d1e449e6a7c0c613a5641b53', 'request_id': 'req-be7cbb54-60fe-4860-81de-cddd8a2372fc'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:32.967867 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be7cbb54-60fe-4860-81de-cddd8a2372fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9566afa9d1e449e6a7c0c613a5641b53', 'request_id': 'req-be7cbb54-60fe-4860-81de-cddd8a2372fc'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:32.973674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be7cbb54-60fe-4860-81de-cddd8a2372fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9566afa9d1e449e6a7c0c613a5641b53', 'request_id': 'req-be7cbb54-60fe-4860-81de-cddd8a2372fc'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:33.052475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be7cbb54-60fe-4860-81de-cddd8a2372fc admin admin] The token cache is being invalidate because project 9566afa9d1e449e6a7c0c613a5641b53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:56:33.052771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be7cbb54-60fe-4860-81de-cddd8a2372fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:33.053394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be7cbb54-60fe-4860-81de-cddd8a2372fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:33.054881 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1241/2456] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:56:32 2022] DELETE /identity/v3/projects/9566afa9d1e449e6a7c0c613a5641b53 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:33.058959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd5934-fc84-49b5-a450-e2aab25c374e service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:33.068679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd5934-fc84-49b5-a450-e2aab25c374e service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:33.124328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f81dfb-0201-496c-9dba-fe58512daaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:33.124978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f81dfb-0201-496c-9dba-fe58512daaaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:33.125261 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f81dfb-0201-496c-9dba-fe58512daaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:33.125538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f81dfb-0201-496c-9dba-fe58512daaaf admin admin] PATH_INFO: `/v3/users/f0f8abed64664e41b4a4042bf388a466` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:33.126674 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f81dfb-0201-496c-9dba-fe58512daaaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0f8abed64664e41b4a4042bf388a466)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:33.128627 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f81dfb-0201-496c-9dba-fe58512daaaf admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:33.341799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a3f81dfb-0201-496c-9dba-fe58512daaaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0f8abed64664e41b4a4042bf388a466'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:33.342946 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a3f81dfb-0201-496c-9dba-fe58512daaaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0f8abed64664e41b4a4042bf388a466'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:33.343558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a3f81dfb-0201-496c-9dba-fe58512daaaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0f8abed64664e41b4a4042bf388a466'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:33.350532 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a3f81dfb-0201-496c-9dba-fe58512daaaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0f8abed64664e41b4a4042bf388a466'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:33.426047 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a3f81dfb-0201-496c-9dba-fe58512daaaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0f8abed64664e41b4a4042bf388a466'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:33.437505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:33.445130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1216/2457] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:56:33 2022] DELETE /identity/v3/users/f0f8abed64664e41b4a4042bf388a466 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:33.832095 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cbb54-60fe-4860-81de-cddd8a2372fc admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:33.833586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-be7cbb54-60fe-4860-81de-cddd8a2372fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:33.847653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cbb54-60fe-4860-81de-cddd8a2372fc admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:33.914687 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb5390-46de-461e-93c9-03cb30147017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:33.915556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb5390-46de-461e-93c9-03cb30147017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:33.915935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb5390-46de-461e-93c9-03cb30147017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:33.916272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb5390-46de-461e-93c9-03cb30147017 admin admin] PATH_INFO: `/v3/projects/c5c675d8b33245688707ea63f5faca08` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:33.917625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb5390-46de-461e-93c9-03cb30147017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5c675d8b33245688707ea63f5faca08)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:33.920221 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb5390-46de-461e-93c9-03cb30147017 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:34.053135 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4acb5390-46de-461e-93c9-03cb30147017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5c675d8b33245688707ea63f5faca08', 'request_id': 'req-4acb5390-46de-461e-93c9-03cb30147017'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:34.059286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4acb5390-46de-461e-93c9-03cb30147017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5c675d8b33245688707ea63f5faca08', 'request_id': 'req-4acb5390-46de-461e-93c9-03cb30147017'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:34.064905 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4acb5390-46de-461e-93c9-03cb30147017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5c675d8b33245688707ea63f5faca08', 'request_id': 'req-4acb5390-46de-461e-93c9-03cb30147017'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:34.129969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:34.134479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4acb5390-46de-461e-93c9-03cb30147017 admin admin] The token cache is being invalidate because project c5c675d8b33245688707ea63f5faca08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:56:34.134769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4acb5390-46de-461e-93c9-03cb30147017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:34.135387 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4acb5390-46de-461e-93c9-03cb30147017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:34.137023 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1242/2458] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:56:33 2022] DELETE /identity/v3/projects/c5c675d8b33245688707ea63f5faca08 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:56:34.142425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ea278-6be9-4222-92f3-7e573678123d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:34.142899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ea278-6be9-4222-92f3-7e573678123d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:34.143016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ea278-6be9-4222-92f3-7e573678123d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:34.162440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-4a2ea278-6be9-4222-92f3-7e573678123d None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:34.163616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-4a2ea278-6be9-4222-92f3-7e573678123d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:34.221446 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1217/2459] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:34.229466 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e68c0-c9ae-4822-89c3-a6c78efdf269 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:34.229956 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e68c0-c9ae-4822-89c3-a6c78efdf269 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:34.230456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e68c0-c9ae-4822-89c3-a6c78efdf269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:34.250420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a09e68c0-c9ae-4822-89c3-a6c78efdf269 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:34.251730 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a09e68c0-c9ae-4822-89c3-a6c78efdf269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:34.260202 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1243/2460] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:34.264642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ea278-6be9-4222-92f3-7e573678123d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:34.274199 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ea278-6be9-4222-92f3-7e573678123d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:34.278616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802d686-3258-4bd9-8a08-37969c9a60b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:34.279203 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802d686-3258-4bd9-8a08-37969c9a60b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:34.279479 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802d686-3258-4bd9-8a08-37969c9a60b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:34.279748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802d686-3258-4bd9-8a08-37969c9a60b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:34.280379 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802d686-3258-4bd9-8a08-37969c9a60b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:56:34.280707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802d686-3258-4bd9-8a08-37969c9a60b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:34.282303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802d686-3258-4bd9-8a08-37969c9a60b9 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:34.293503 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1218/2461] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:56:34.297663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e68c0-c9ae-4822-89c3-a6c78efdf269 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:34.306733 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e68c0-c9ae-4822-89c3-a6c78efdf269 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:34.311820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f0903-3583-4c7f-8ee1-211748af3f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:34.312408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f0903-3583-4c7f-8ee1-211748af3f8e admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:34.312714 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f0903-3583-4c7f-8ee1-211748af3f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:34.312984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f0903-3583-4c7f-8ee1-211748af3f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:34.313713 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f0903-3583-4c7f-8ee1-211748af3f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:34.315601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f0903-3583-4c7f-8ee1-211748af3f8e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:34.404172 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1244/2462] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:56:34 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:56:34.408565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802d686-3258-4bd9-8a08-37969c9a60b9 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:34.418425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802d686-3258-4bd9-8a08-37969c9a60b9 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:34.473677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a422895-8d26-4d8b-809f-b72a203670b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:34.474273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a422895-8d26-4d8b-809f-b72a203670b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:34.474549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a422895-8d26-4d8b-809f-b72a203670b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:34.474822 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a422895-8d26-4d8b-809f-b72a203670b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:34.475443 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a422895-8d26-4d8b-809f-b72a203670b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:34.477449 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a422895-8d26-4d8b-809f-b72a203670b2 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:34.569581 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1219/2463] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:56:34 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:56:34.573815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f0903-3583-4c7f-8ee1-211748af3f8e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:34.584746 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f0903-3583-4c7f-8ee1-211748af3f8e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:34.589373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82f8ef-2c2c-42bd-b64b-54aa7aed5f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:34.589954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82f8ef-2c2c-42bd-b64b-54aa7aed5f04 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:34.590239 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82f8ef-2c2c-42bd-b64b-54aa7aed5f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:34.590512 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82f8ef-2c2c-42bd-b64b-54aa7aed5f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:34.591152 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d82f8ef-2c2c-42bd-b64b-54aa7aed5f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:34.592834 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d82f8ef-2c2c-42bd-b64b-54aa7aed5f04 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:34.602631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1245/2464] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:56:34.606485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a422895-8d26-4d8b-809f-b72a203670b2 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:34.616930 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a422895-8d26-4d8b-809f-b72a203670b2 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:34.621531 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253460ad-3c87-4f62-b047-06b86b2ab9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:34.622115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253460ad-3c87-4f62-b047-06b86b2ab9ad admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:34.622389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253460ad-3c87-4f62-b047-06b86b2ab9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:34.622656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253460ad-3c87-4f62-b047-06b86b2ab9ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:34.623964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253460ad-3c87-4f62-b047-06b86b2ab9ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:34.625645 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253460ad-3c87-4f62-b047-06b86b2ab9ad admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:34.636173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1220/2465] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:56:34.640220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82f8ef-2c2c-42bd-b64b-54aa7aed5f04 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:34.649897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82f8ef-2c2c-42bd-b64b-54aa7aed5f04 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:34.654447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377d02f-afc2-4e66-a664-65936c58ccb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:34.655024 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377d02f-afc2-4e66-a664-65936c58ccb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:34.655299 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377d02f-afc2-4e66-a664-65936c58ccb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:34.655570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377d02f-afc2-4e66-a664-65936c58ccb1 admin admin] PATH_INFO: `/v3/projects/56e6aaf9a3cc400694affbfcf147c40f/users/fe414010a3e349f99977929c5c03e544/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:34.669581 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377d02f-afc2-4e66-a664-65936c58ccb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56e6aaf9a3cc400694affbfcf147c40f, user_id=fe414010a3e349f99977929c5c03e544, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:34.672729 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377d02f-afc2-4e66-a664-65936c58ccb1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:34.751974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1246/2466] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:56:34 2022] PUT /identity/v3/projects/56e6aaf9a3cc400694affbfcf147c40f/users/fe414010a3e349f99977929c5c03e544/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:34.763516 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d80e92-48a4-474f-93ce-5b35356ccf53 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:34.763824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d80e92-48a4-474f-93ce-5b35356ccf53 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:34.764103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d80e92-48a4-474f-93ce-5b35356ccf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:34.792341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-a2d80e92-48a4-474f-93ce-5b35356ccf53 None None] MFA Rules not processed for user `fe414010a3e349f99977929c5c03e544`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:34.793575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a2d80e92-48a4-474f-93ce-5b35356ccf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:34.839471 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:34.856326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1221/2467] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:34.861336 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377d02f-afc2-4e66-a664-65936c58ccb1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:34.921571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377d02f-afc2-4e66-a664-65936c58ccb1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:34.926095 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39446f3e-3c67-479f-8f1d-81aefc44d370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:34.926692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39446f3e-3c67-479f-8f1d-81aefc44d370 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:34.926971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39446f3e-3c67-479f-8f1d-81aefc44d370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:34.927243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39446f3e-3c67-479f-8f1d-81aefc44d370 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:34.927871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39446f3e-3c67-479f-8f1d-81aefc44d370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:34.929788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39446f3e-3c67-479f-8f1d-81aefc44d370 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:35.038279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1247/2468] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:56:34 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:56:35.042422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d80e92-48a4-474f-93ce-5b35356ccf53 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:35.054360 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d80e92-48a4-474f-93ce-5b35356ccf53 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:35.109327 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239a2d0-b63d-47fd-99af-dfe6d445278c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:35.109931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239a2d0-b63d-47fd-99af-dfe6d445278c admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:35.110212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239a2d0-b63d-47fd-99af-dfe6d445278c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:35.110504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239a2d0-b63d-47fd-99af-dfe6d445278c admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:35.111114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239a2d0-b63d-47fd-99af-dfe6d445278c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:35.113049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239a2d0-b63d-47fd-99af-dfe6d445278c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:35.220219 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1222/2469] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:56:35 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:56:35.224267 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39446f3e-3c67-479f-8f1d-81aefc44d370 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:35.235035 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39446f3e-3c67-479f-8f1d-81aefc44d370 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:35.244907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7ec6d-0464-4403-a14b-7173d3ed45f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:35.244907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7ec6d-0464-4403-a14b-7173d3ed45f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:35.244907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7ec6d-0464-4403-a14b-7173d3ed45f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:35.244907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7ec6d-0464-4403-a14b-7173d3ed45f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:35.245448 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7ec6d-0464-4403-a14b-7173d3ed45f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:35.245448 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7ec6d-0464-4403-a14b-7173d3ed45f8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:35.253547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1248/2470] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:56:35.257746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239a2d0-b63d-47fd-99af-dfe6d445278c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:35.267651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239a2d0-b63d-47fd-99af-dfe6d445278c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:35.272227 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c50aec-22f8-4500-b736-343868b5edac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:35.272852 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c50aec-22f8-4500-b736-343868b5edac admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:35.273131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c50aec-22f8-4500-b736-343868b5edac admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:35.273400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c50aec-22f8-4500-b736-343868b5edac admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:35.274032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c50aec-22f8-4500-b736-343868b5edac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:35.275604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c50aec-22f8-4500-b736-343868b5edac admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:35.285835 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1223/2471] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:56:35.289840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7ec6d-0464-4403-a14b-7173d3ed45f8 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:35.300138 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7ec6d-0464-4403-a14b-7173d3ed45f8 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:35.304773 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f98c8-8aa7-49d0-9b7b-8c0907b28c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:35.305420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f98c8-8aa7-49d0-9b7b-8c0907b28c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:35.305700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f98c8-8aa7-49d0-9b7b-8c0907b28c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:35.305972 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f98c8-8aa7-49d0-9b7b-8c0907b28c97 admin admin] PATH_INFO: `/v3/projects/ab0c9ab5e484401d8798dd1ca5af50b9/users/315257d27084430890cd198d5586b2ff/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:35.319906 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f98c8-8aa7-49d0-9b7b-8c0907b28c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0c9ab5e484401d8798dd1ca5af50b9, user_id=315257d27084430890cd198d5586b2ff, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:35.322620 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f98c8-8aa7-49d0-9b7b-8c0907b28c97 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:35.436675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1249/2472] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:56:35 2022] PUT /identity/v3/projects/ab0c9ab5e484401d8798dd1ca5af50b9/users/315257d27084430890cd198d5586b2ff/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:56:35.447294 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610f40f-965a-4aea-abb4-7fa201fb3213 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:35.447630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610f40f-965a-4aea-abb4-7fa201fb3213 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:35.447862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610f40f-965a-4aea-abb4-7fa201fb3213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:35.476618 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-1610f40f-965a-4aea-abb4-7fa201fb3213 None None] MFA Rules not processed for user `315257d27084430890cd198d5586b2ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:35.477793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1610f40f-965a-4aea-abb4-7fa201fb3213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:35.553316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1224/2473] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:35.558060 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f98c8-8aa7-49d0-9b7b-8c0907b28c97 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:35.603808 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:35.619378 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f98c8-8aa7-49d0-9b7b-8c0907b28c97 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:35.626146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e502bf-e2ee-4154-b86f-e4ddc67168c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:35.626146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e502bf-e2ee-4154-b86f-e4ddc67168c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:35.626146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e502bf-e2ee-4154-b86f-e4ddc67168c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:35.626146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e502bf-e2ee-4154-b86f-e4ddc67168c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:35.626546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e502bf-e2ee-4154-b86f-e4ddc67168c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:35.627736 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e502bf-e2ee-4154-b86f-e4ddc67168c1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:35.716804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1250/2474] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:56:35 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:56:35.720843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610f40f-965a-4aea-abb4-7fa201fb3213 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:35.733170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610f40f-965a-4aea-abb4-7fa201fb3213 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:35.787122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:35.791797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6484b-2733-4ed5-a5b3-9321ab6aed74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:35.792483 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6484b-2733-4ed5-a5b3-9321ab6aed74 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:35.792762 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6484b-2733-4ed5-a5b3-9321ab6aed74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:35.793049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6484b-2733-4ed5-a5b3-9321ab6aed74 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:35.793657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6484b-2733-4ed5-a5b3-9321ab6aed74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:35.795565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6484b-2733-4ed5-a5b3-9321ab6aed74 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:35.924068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1225/2475] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:56:35 2022] POST /identity/v3/users => generated 299 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:56:35.928002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e502bf-e2ee-4154-b86f-e4ddc67168c1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:35.938563 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e502bf-e2ee-4154-b86f-e4ddc67168c1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:35.943053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9528bd-2ba4-467f-9bd4-67285a3b8201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:35.943647 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9528bd-2ba4-467f-9bd4-67285a3b8201 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:35.943921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9528bd-2ba4-467f-9bd4-67285a3b8201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:35.944189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9528bd-2ba4-467f-9bd4-67285a3b8201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:35.944934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9528bd-2ba4-467f-9bd4-67285a3b8201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:35.946494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9528bd-2ba4-467f-9bd4-67285a3b8201 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:35.956862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1251/2476] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:56:35.961113 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6484b-2733-4ed5-a5b3-9321ab6aed74 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:35.971137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6484b-2733-4ed5-a5b3-9321ab6aed74 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:35.975815 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd16679-d50b-46c5-ab52-6e6430c53f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:35.976506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd16679-d50b-46c5-ab52-6e6430c53f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:35.976793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd16679-d50b-46c5-ab52-6e6430c53f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:35.977072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd16679-d50b-46c5-ab52-6e6430c53f80 admin admin] PATH_INFO: `/v3/projects/01a1f6a2ed9a4476a0b523649feb262c/users/4b2387eaf1584bcda105b26079bade7e/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:35.991486 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd16679-d50b-46c5-ab52-6e6430c53f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01a1f6a2ed9a4476a0b523649feb262c, user_id=4b2387eaf1584bcda105b26079bade7e, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:35.994122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd16679-d50b-46c5-ab52-6e6430c53f80 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:36.064613 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1226/2477] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:56:35 2022] PUT /identity/v3/projects/01a1f6a2ed9a4476a0b523649feb262c/users/4b2387eaf1584bcda105b26079bade7e/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:36.068786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9528bd-2ba4-467f-9bd4-67285a3b8201 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:36.079281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9528bd-2ba4-467f-9bd4-67285a3b8201 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:36.134398 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c67930-4224-4f3b-b056-580759c179c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:36.134999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c67930-4224-4f3b-b056-580759c179c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:36.135279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c67930-4224-4f3b-b056-580759c179c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:36.135554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c67930-4224-4f3b-b056-580759c179c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:36.136189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c67930-4224-4f3b-b056-580759c179c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:36.137818 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c67930-4224-4f3b-b056-580759c179c2 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:36.147220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1252/2478] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:56:36.151373 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd16679-d50b-46c5-ab52-6e6430c53f80 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:36.163449 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd16679-d50b-46c5-ab52-6e6430c53f80 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:36.168298 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d422c4-d9e5-48fe-baf7-1c9bb3a2d56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:36.168987 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d422c4-d9e5-48fe-baf7-1c9bb3a2d56b admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:36.169296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d422c4-d9e5-48fe-baf7-1c9bb3a2d56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:36.169571 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d422c4-d9e5-48fe-baf7-1c9bb3a2d56b admin admin] PATH_INFO: `/v3/projects/01a1f6a2ed9a4476a0b523649feb262c/users/4b2387eaf1584bcda105b26079bade7e/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:36.172094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d422c4-d9e5-48fe-baf7-1c9bb3a2d56b admin admin] RBAC: Authorizing `identity:create_grant(project_id=01a1f6a2ed9a4476a0b523649feb262c, user_id=4b2387eaf1584bcda105b26079bade7e, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:36.174774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d422c4-d9e5-48fe-baf7-1c9bb3a2d56b admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:36.185342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1227/2479] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:56:36 2022] PUT /identity/v3/projects/01a1f6a2ed9a4476a0b523649feb262c/users/4b2387eaf1584bcda105b26079bade7e/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:36.197492 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6439858-2fd6-422c-bb00-f6508cdff2d8 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:36.197781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6439858-2fd6-422c-bb00-f6508cdff2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:36.198057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6439858-2fd6-422c-bb00-f6508cdff2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:36.229892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b6439858-2fd6-422c-bb00-f6508cdff2d8 None None] MFA Rules not processed for user `4b2387eaf1584bcda105b26079bade7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:36.231107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b6439858-2fd6-422c-bb00-f6508cdff2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:36.297783 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1253/2480] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:36.308312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d422c4-d9e5-48fe-baf7-1c9bb3a2d56b admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:36.310119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c7d422c4-d9e5-48fe-baf7-1c9bb3a2d56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:36.367845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d422c4-d9e5-48fe-baf7-1c9bb3a2d56b admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:36.371450 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9e1d0-01b4-4513-8261-529730d3aca4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:36.372041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9e1d0-01b4-4513-8261-529730d3aca4 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:36.372321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9e1d0-01b4-4513-8261-529730d3aca4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:36.372623 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9e1d0-01b4-4513-8261-529730d3aca4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:36.382288 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9e1d0-01b4-4513-8261-529730d3aca4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:36.383698 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9e1d0-01b4-4513-8261-529730d3aca4 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:36.392896 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1228/2481] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:56:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:56:37.396001 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6439858-2fd6-422c-bb00-f6508cdff2d8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:37.397426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b6439858-2fd6-422c-bb00-f6508cdff2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:37.409609 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6439858-2fd6-422c-bb00-f6508cdff2d8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:37.413132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdd347-1ae1-47b9-a919-53275506778d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:37.413732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdd347-1ae1-47b9-a919-53275506778d service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:37.422735 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdd347-1ae1-47b9-a919-53275506778d service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:37.422735 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdd347-1ae1-47b9-a919-53275506778d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:37.453427 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:37.471912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdd347-1ae1-47b9-a919-53275506778d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:37.473382 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdd347-1ae1-47b9-a919-53275506778d service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:37.482048 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1254/2482] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:56:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:56:43.647303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9e1d0-01b4-4513-8261-529730d3aca4 service cinder] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:43.655957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9e1d0-01b4-4513-8261-529730d3aca4 service cinder] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:43.660340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6431a-b2fa-4b6e-afbd-937fd448365b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:43.660958 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6431a-b2fa-4b6e-afbd-937fd448365b service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:43.661239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6431a-b2fa-4b6e-afbd-937fd448365b service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:43.661510 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6431a-b2fa-4b6e-afbd-937fd448365b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:43.716654 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6431a-b2fa-4b6e-afbd-937fd448365b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:43.718079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6431a-b2fa-4b6e-afbd-937fd448365b service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:43.727325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1229/2483] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:56:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:56:47.545777 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=10 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:48.119698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdd347-1ae1-47b9-a919-53275506778d service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:48.179903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdd347-1ae1-47b9-a919-53275506778d service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:48.184216 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512fa03-c0c8-48b9-9cd9-bbf898eded1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:48.184856 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512fa03-c0c8-48b9-9cd9-bbf898eded1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:48.185134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512fa03-c0c8-48b9-9cd9-bbf898eded1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:48.185405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512fa03-c0c8-48b9-9cd9-bbf898eded1a admin admin] PATH_INFO: `/v3/users/fe414010a3e349f99977929c5c03e544` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:48.186496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512fa03-c0c8-48b9-9cd9-bbf898eded1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe414010a3e349f99977929c5c03e544)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:48.188394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512fa03-c0c8-48b9-9cd9-bbf898eded1a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:48.373072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9512fa03-c0c8-48b9-9cd9-bbf898eded1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe414010a3e349f99977929c5c03e544'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:48.373822 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9512fa03-c0c8-48b9-9cd9-bbf898eded1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe414010a3e349f99977929c5c03e544'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:48.374434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9512fa03-c0c8-48b9-9cd9-bbf898eded1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe414010a3e349f99977929c5c03e544'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:48.443243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9512fa03-c0c8-48b9-9cd9-bbf898eded1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe414010a3e349f99977929c5c03e544'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:48.451177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9512fa03-c0c8-48b9-9cd9-bbf898eded1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe414010a3e349f99977929c5c03e544'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:48.468970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1255/2484] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:56:48 2022] DELETE /identity/v3/users/fe414010a3e349f99977929c5c03e544 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:48.474090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbeb69-2d36-4bd1-a78e-46f1a92b5207 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:48.476011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbeb69-2d36-4bd1-a78e-46f1a92b5207 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:48.476011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbeb69-2d36-4bd1-a78e-46f1a92b5207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:48.493916 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-fcdbeb69-2d36-4bd1-a78e-46f1a92b5207 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:48.495150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-fcdbeb69-2d36-4bd1-a78e-46f1a92b5207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:48.545304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:48.560601 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1230/2485] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:48.570863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512fa03-c0c8-48b9-9cd9-bbf898eded1a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:48.572373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9512fa03-c0c8-48b9-9cd9-bbf898eded1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:48.633102 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512fa03-c0c8-48b9-9cd9-bbf898eded1a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:48.636814 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01c8fe-97b3-4f20-9970-cfa3d291b7d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:48.637425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01c8fe-97b3-4f20-9970-cfa3d291b7d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:48.637701 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01c8fe-97b3-4f20-9970-cfa3d291b7d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:48.638969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01c8fe-97b3-4f20-9970-cfa3d291b7d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:48.647367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01c8fe-97b3-4f20-9970-cfa3d291b7d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:48.648862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01c8fe-97b3-4f20-9970-cfa3d291b7d4 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:48.659102 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1256/2486] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:56:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:56:49.096714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbeb69-2d36-4bd1-a78e-46f1a92b5207 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:49.098888 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-fcdbeb69-2d36-4bd1-a78e-46f1a92b5207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:49.110530 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbeb69-2d36-4bd1-a78e-46f1a92b5207 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:49.115233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abce3d-350f-4c18-902e-f21efc8b3345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:49.115907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abce3d-350f-4c18-902e-f21efc8b3345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:49.116150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abce3d-350f-4c18-902e-f21efc8b3345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:49.116456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abce3d-350f-4c18-902e-f21efc8b3345 admin admin] PATH_INFO: `/v3/projects/56e6aaf9a3cc400694affbfcf147c40f` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:49.117595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abce3d-350f-4c18-902e-f21efc8b3345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56e6aaf9a3cc400694affbfcf147c40f)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:49.119509 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abce3d-350f-4c18-902e-f21efc8b3345 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:49.255921 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-91abce3d-350f-4c18-902e-f21efc8b3345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56e6aaf9a3cc400694affbfcf147c40f', 'request_id': 'req-91abce3d-350f-4c18-902e-f21efc8b3345'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:49.262528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-91abce3d-350f-4c18-902e-f21efc8b3345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56e6aaf9a3cc400694affbfcf147c40f', 'request_id': 'req-91abce3d-350f-4c18-902e-f21efc8b3345'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:49.268388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-91abce3d-350f-4c18-902e-f21efc8b3345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56e6aaf9a3cc400694affbfcf147c40f', 'request_id': 'req-91abce3d-350f-4c18-902e-f21efc8b3345'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:49.317660 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-91abce3d-350f-4c18-902e-f21efc8b3345 admin admin] The token cache is being invalidate because project 56e6aaf9a3cc400694affbfcf147c40f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:56:49.317951 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-91abce3d-350f-4c18-902e-f21efc8b3345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:49.318598 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-91abce3d-350f-4c18-902e-f21efc8b3345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:49.320072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1231/2487] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:56:49 2022] DELETE /identity/v3/projects/56e6aaf9a3cc400694affbfcf147c40f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:56:49.324101 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01c8fe-97b3-4f20-9970-cfa3d291b7d4 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:49.334443 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01c8fe-97b3-4f20-9970-cfa3d291b7d4 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:49.353601 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:49.391332 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc0ee4-4dc7-4cc3-b7e2-6705ea89d694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:49.392639 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc0ee4-4dc7-4cc3-b7e2-6705ea89d694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:49.393151 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc0ee4-4dc7-4cc3-b7e2-6705ea89d694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:49.393450 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc0ee4-4dc7-4cc3-b7e2-6705ea89d694 admin admin] PATH_INFO: `/v3/users/315257d27084430890cd198d5586b2ff` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:49.395006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc0ee4-4dc7-4cc3-b7e2-6705ea89d694 admin admin] RBAC: Authorizing `identity:delete_user(user_id=315257d27084430890cd198d5586b2ff)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:49.397042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc0ee4-4dc7-4cc3-b7e2-6705ea89d694 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:49.626643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-14bc0ee4-4dc7-4cc3-b7e2-6705ea89d694 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '315257d27084430890cd198d5586b2ff'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:49.627820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-14bc0ee4-4dc7-4cc3-b7e2-6705ea89d694 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '315257d27084430890cd198d5586b2ff'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:49.628452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-14bc0ee4-4dc7-4cc3-b7e2-6705ea89d694 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '315257d27084430890cd198d5586b2ff'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:49.711721 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-14bc0ee4-4dc7-4cc3-b7e2-6705ea89d694 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '315257d27084430890cd198d5586b2ff'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:49.718847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-14bc0ee4-4dc7-4cc3-b7e2-6705ea89d694 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '315257d27084430890cd198d5586b2ff'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:49.736039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1257/2488] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:56:49 2022] DELETE /identity/v3/users/315257d27084430890cd198d5586b2ff => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:50.158496 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abce3d-350f-4c18-902e-f21efc8b3345 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:50.161036 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-91abce3d-350f-4c18-902e-f21efc8b3345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:50.177636 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abce3d-350f-4c18-902e-f21efc8b3345 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:50.237103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1494885-ec02-4e70-86f1-533d68077df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:50.240932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1494885-ec02-4e70-86f1-533d68077df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:50.240932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1494885-ec02-4e70-86f1-533d68077df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:50.240932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1494885-ec02-4e70-86f1-533d68077df1 admin admin] PATH_INFO: `/v3/projects/ab0c9ab5e484401d8798dd1ca5af50b9` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:50.240932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1494885-ec02-4e70-86f1-533d68077df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0c9ab5e484401d8798dd1ca5af50b9)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:50.241347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1494885-ec02-4e70-86f1-533d68077df1 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:50.376916 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a1494885-ec02-4e70-86f1-533d68077df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0c9ab5e484401d8798dd1ca5af50b9', 'request_id': 'req-a1494885-ec02-4e70-86f1-533d68077df1'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:50.383458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a1494885-ec02-4e70-86f1-533d68077df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0c9ab5e484401d8798dd1ca5af50b9', 'request_id': 'req-a1494885-ec02-4e70-86f1-533d68077df1'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:50.389535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a1494885-ec02-4e70-86f1-533d68077df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0c9ab5e484401d8798dd1ca5af50b9', 'request_id': 'req-a1494885-ec02-4e70-86f1-533d68077df1'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:50.500687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a1494885-ec02-4e70-86f1-533d68077df1 admin admin] The token cache is being invalidate because project ab0c9ab5e484401d8798dd1ca5af50b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:56:50.500879 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a1494885-ec02-4e70-86f1-533d68077df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:50.501505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a1494885-ec02-4e70-86f1-533d68077df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:50.503034 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1232/2489] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:56:50 2022] DELETE /identity/v3/projects/ab0c9ab5e484401d8798dd1ca5af50b9 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:50.507356 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc0ee4-4dc7-4cc3-b7e2-6705ea89d694 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:50.508913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-14bc0ee4-4dc7-4cc3-b7e2-6705ea89d694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:50.522664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc0ee4-4dc7-4cc3-b7e2-6705ea89d694 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:50.579320 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded735a5-92f0-43bf-8bd4-591517f759cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:50.580464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded735a5-92f0-43bf-8bd4-591517f759cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:50.580745 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded735a5-92f0-43bf-8bd4-591517f759cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:50.581014 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded735a5-92f0-43bf-8bd4-591517f759cb admin admin] PATH_INFO: `/v3/users/4b2387eaf1584bcda105b26079bade7e` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:50.582129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded735a5-92f0-43bf-8bd4-591517f759cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b2387eaf1584bcda105b26079bade7e)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:50.584120 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded735a5-92f0-43bf-8bd4-591517f759cb admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:50.767919 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:50.770314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ded735a5-92f0-43bf-8bd4-591517f759cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b2387eaf1584bcda105b26079bade7e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:50.770957 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ded735a5-92f0-43bf-8bd4-591517f759cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b2387eaf1584bcda105b26079bade7e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:50.771559 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ded735a5-92f0-43bf-8bd4-591517f759cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b2387eaf1584bcda105b26079bade7e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:50.866417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ded735a5-92f0-43bf-8bd4-591517f759cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b2387eaf1584bcda105b26079bade7e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:50.873676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ded735a5-92f0-43bf-8bd4-591517f759cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b2387eaf1584bcda105b26079bade7e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:50.891459 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1258/2490] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:56:50 2022] DELETE /identity/v3/users/4b2387eaf1584bcda105b26079bade7e => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:51.346365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1494885-ec02-4e70-86f1-533d68077df1 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:51.347841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a1494885-ec02-4e70-86f1-533d68077df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:51.362524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:51.362524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1494885-ec02-4e70-86f1-533d68077df1 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:51.418963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535cd63c-5096-450a-a798-3a57291d5196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:51.419684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535cd63c-5096-450a-a798-3a57291d5196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:51.419966 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535cd63c-5096-450a-a798-3a57291d5196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:51.420239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535cd63c-5096-450a-a798-3a57291d5196 admin admin] PATH_INFO: `/v3/projects/01a1f6a2ed9a4476a0b523649feb262c` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:51.421381 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535cd63c-5096-450a-a798-3a57291d5196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01a1f6a2ed9a4476a0b523649feb262c)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:51.423278 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535cd63c-5096-450a-a798-3a57291d5196 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:51.564746 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-535cd63c-5096-450a-a798-3a57291d5196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01a1f6a2ed9a4476a0b523649feb262c', 'request_id': 'req-535cd63c-5096-450a-a798-3a57291d5196'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:51.571714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-535cd63c-5096-450a-a798-3a57291d5196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01a1f6a2ed9a4476a0b523649feb262c', 'request_id': 'req-535cd63c-5096-450a-a798-3a57291d5196'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:51.577265 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-535cd63c-5096-450a-a798-3a57291d5196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01a1f6a2ed9a4476a0b523649feb262c', 'request_id': 'req-535cd63c-5096-450a-a798-3a57291d5196'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:51.648285 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-535cd63c-5096-450a-a798-3a57291d5196 admin admin] The token cache is being invalidate because project 01a1f6a2ed9a4476a0b523649feb262c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:56:51.648836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-535cd63c-5096-450a-a798-3a57291d5196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:51.649868 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-535cd63c-5096-450a-a798-3a57291d5196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:56:51.651667 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1233/2491] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:56:51 2022] DELETE /identity/v3/projects/01a1f6a2ed9a4476a0b523649feb262c => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:51.659273 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178facc1-36ec-4652-9e10-6e93d4f485db None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:51.659567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178facc1-36ec-4652-9e10-6e93d4f485db None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:51.659844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178facc1-36ec-4652-9e10-6e93d4f485db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:51.680626 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-178facc1-36ec-4652-9e10-6e93d4f485db None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:51.681835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-178facc1-36ec-4652-9e10-6e93d4f485db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:51.744853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1259/2492] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:51.753607 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64a195-6387-4508-9ca2-e3a802edcca0 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:51.753895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64a195-6387-4508-9ca2-e3a802edcca0 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:51.754203 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64a195-6387-4508-9ca2-e3a802edcca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:51.776276 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-3c64a195-6387-4508-9ca2-e3a802edcca0 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:51.776276 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3c64a195-6387-4508-9ca2-e3a802edcca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:51.784806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1234/2493] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:51.789163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178facc1-36ec-4652-9e10-6e93d4f485db None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:51.799194 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178facc1-36ec-4652-9e10-6e93d4f485db None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:51.803839 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f6f80-871b-4c1e-9716-361dde41b3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:51.804478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f6f80-871b-4c1e-9716-361dde41b3fc admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:51.804763 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f6f80-871b-4c1e-9716-361dde41b3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:51.805039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f6f80-871b-4c1e-9716-361dde41b3fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:51.805691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f6f80-871b-4c1e-9716-361dde41b3fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:56:51.806115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f6f80-871b-4c1e-9716-361dde41b3fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:51.807715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f6f80-871b-4c1e-9716-361dde41b3fc admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:51.819447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1260/2494] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:56:51.823680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64a195-6387-4508-9ca2-e3a802edcca0 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:51.833548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64a195-6387-4508-9ca2-e3a802edcca0 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:51.838223 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af509da-e2a6-4a4b-ba35-9527c0e6e0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:51.838811 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af509da-e2a6-4a4b-ba35-9527c0e6e0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:51.839091 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af509da-e2a6-4a4b-ba35-9527c0e6e0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:51.839357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af509da-e2a6-4a4b-ba35-9527c0e6e0e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:51.839973 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af509da-e2a6-4a4b-ba35-9527c0e6e0e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:51.841871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af509da-e2a6-4a4b-ba35-9527c0e6e0e8 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:51.922242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1235/2495] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:56:51 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:56:51.926262 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f6f80-871b-4c1e-9716-361dde41b3fc admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:51.936041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f6f80-871b-4c1e-9716-361dde41b3fc admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:51.993904 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c5f6c-e9b8-4fd2-90c6-ea03118ff202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:51.994506 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c5f6c-e9b8-4fd2-90c6-ea03118ff202 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:51.994785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c5f6c-e9b8-4fd2-90c6-ea03118ff202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:51.995064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c5f6c-e9b8-4fd2-90c6-ea03118ff202 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:51.995689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c5f6c-e9b8-4fd2-90c6-ea03118ff202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:51.997632 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c5f6c-e9b8-4fd2-90c6-ea03118ff202 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:52.077642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1261/2496] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:56:51 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:56:52.082654 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af509da-e2a6-4a4b-ba35-9527c0e6e0e8 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:52.093454 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af509da-e2a6-4a4b-ba35-9527c0e6e0e8 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:52.098829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0184b-bab7-4a2c-8027-ce0c992e9a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:52.100244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0184b-bab7-4a2c-8027-ce0c992e9a9b admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:52.100244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0184b-bab7-4a2c-8027-ce0c992e9a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:52.100244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0184b-bab7-4a2c-8027-ce0c992e9a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:52.100666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0184b-bab7-4a2c-8027-ce0c992e9a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:52.102248 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0184b-bab7-4a2c-8027-ce0c992e9a9b admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:52.112169 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1236/2497] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:56:52.116109 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c5f6c-e9b8-4fd2-90c6-ea03118ff202 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:52.125996 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c5f6c-e9b8-4fd2-90c6-ea03118ff202 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:52.131464 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91470d81-3613-44ba-bec3-92babf6cce3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:52.132064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91470d81-3613-44ba-bec3-92babf6cce3c admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:52.132348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91470d81-3613-44ba-bec3-92babf6cce3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:52.132656 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91470d81-3613-44ba-bec3-92babf6cce3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:52.133302 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91470d81-3613-44ba-bec3-92babf6cce3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:52.134964 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91470d81-3613-44ba-bec3-92babf6cce3c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:52.146395 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1262/2498] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:56:52.150519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0184b-bab7-4a2c-8027-ce0c992e9a9b admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:52.161014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0184b-bab7-4a2c-8027-ce0c992e9a9b admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:52.165865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ff181-2004-45c3-b7f6-b32a0c6af9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:52.166894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ff181-2004-45c3-b7f6-b32a0c6af9f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:52.166894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ff181-2004-45c3-b7f6-b32a0c6af9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:52.167260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ff181-2004-45c3-b7f6-b32a0c6af9f6 admin admin] PATH_INFO: `/v3/projects/1d8627531fef472faaf15229336c49de/users/a9ebf98678f54a58b817d7486bfafd7f/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:52.182365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ff181-2004-45c3-b7f6-b32a0c6af9f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d8627531fef472faaf15229336c49de, user_id=a9ebf98678f54a58b817d7486bfafd7f, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:52.185027 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ff181-2004-45c3-b7f6-b32a0c6af9f6 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:52.252786 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1237/2499] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:56:52 2022] PUT /identity/v3/projects/1d8627531fef472faaf15229336c49de/users/a9ebf98678f54a58b817d7486bfafd7f/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:56:52.264107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfaf95-724d-4f02-82b9-0bb07b6feb4a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:52.264395 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfaf95-724d-4f02-82b9-0bb07b6feb4a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:52.264702 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfaf95-724d-4f02-82b9-0bb07b6feb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:52.292457 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e3cfaf95-724d-4f02-82b9-0bb07b6feb4a None None] MFA Rules not processed for user `a9ebf98678f54a58b817d7486bfafd7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:56:52.293649 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e3cfaf95-724d-4f02-82b9-0bb07b6feb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:52.311796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:52.362994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1263/2500] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:56:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:56:52.374619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ff181-2004-45c3-b7f6-b32a0c6af9f6 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:52.376280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-cf1ff181-2004-45c3-b7f6-b32a0c6af9f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:52.411705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:52.434736 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ff181-2004-45c3-b7f6-b32a0c6af9f6 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:52.438366 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac91e67-b010-4862-9903-ad60fdac07c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56418ff737504181a59714aad9f1c375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:52.438997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac91e67-b010-4862-9903-ad60fdac07c3 service glance] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:52.439277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac91e67-b010-4862-9903-ad60fdac07c3 service glance] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:52.439549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac91e67-b010-4862-9903-ad60fdac07c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:52.448239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac91e67-b010-4862-9903-ad60fdac07c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:52.449702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac91e67-b010-4862-9903-ad60fdac07c3 service glance] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:52.458428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1238/2501] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:56:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:57:02.368392 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=20 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:02.506014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=12 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:10.284084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680da01-2ad5-4b5d-9e5f-01af35d8e390 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:10.284750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680da01-2ad5-4b5d-9e5f-01af35d8e390 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:10.284750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680da01-2ad5-4b5d-9e5f-01af35d8e390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:10.304545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0680da01-2ad5-4b5d-9e5f-01af35d8e390 None None] MFA Rules not processed for user `88efd5eb09064aa2aa89e6d9580c6476`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:10.305759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0680da01-2ad5-4b5d-9e5f-01af35d8e390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:10.360733 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1264/2502] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:57:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:57:20.354042 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=25 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:20.378093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac91e67-b010-4862-9903-ad60fdac07c3 service glance] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:20.442242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac91e67-b010-4862-9903-ad60fdac07c3 service glance] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:20.446733 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ebbfd-88b3-46da-8575-5c0ced04a2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:20.447356 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ebbfd-88b3-46da-8575-5c0ced04a2df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:20.447636 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ebbfd-88b3-46da-8575-5c0ced04a2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:20.447910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ebbfd-88b3-46da-8575-5c0ced04a2df admin admin] PATH_INFO: `/v3/users/a9ebf98678f54a58b817d7486bfafd7f` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:20.449139 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ebbfd-88b3-46da-8575-5c0ced04a2df admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9ebf98678f54a58b817d7486bfafd7f)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:20.451104 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ebbfd-88b3-46da-8575-5c0ced04a2df admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:20.620829 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d46ebbfd-88b3-46da-8575-5c0ced04a2df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9ebf98678f54a58b817d7486bfafd7f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:20.621491 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d46ebbfd-88b3-46da-8575-5c0ced04a2df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9ebf98678f54a58b817d7486bfafd7f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:20.622104 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d46ebbfd-88b3-46da-8575-5c0ced04a2df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9ebf98678f54a58b817d7486bfafd7f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:20.629438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d46ebbfd-88b3-46da-8575-5c0ced04a2df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9ebf98678f54a58b817d7486bfafd7f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:20.715337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-d46ebbfd-88b3-46da-8575-5c0ced04a2df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9ebf98678f54a58b817d7486bfafd7f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:20.735372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1239/2503] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:57:20 2022] DELETE /identity/v3/users/a9ebf98678f54a58b817d7486bfafd7f => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:57:20.739863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0fe64-72eb-473f-a54c-8478e8f18352 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:20.740146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0fe64-72eb-473f-a54c-8478e8f18352 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:20.740433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0fe64-72eb-473f-a54c-8478e8f18352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:20.761888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-73d0fe64-72eb-473f-a54c-8478e8f18352 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:20.764616 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-73d0fe64-72eb-473f-a54c-8478e8f18352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:20.821028 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1265/2504] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:57:20.830748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ebbfd-88b3-46da-8575-5c0ced04a2df admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:20.832688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d46ebbfd-88b3-46da-8575-5c0ced04a2df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:20.890011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ebbfd-88b3-46da-8575-5c0ced04a2df admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:20.893604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfd3a4-686f-4c99-aac7-7dad568e85c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:20.894719 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfd3a4-686f-4c99-aac7-7dad568e85c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:20.894997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfd3a4-686f-4c99-aac7-7dad568e85c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:20.895265 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfd3a4-686f-4c99-aac7-7dad568e85c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:20.904026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cfd3a4-686f-4c99-aac7-7dad568e85c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:20.905467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cfd3a4-686f-4c99-aac7-7dad568e85c3 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:20.915225 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1240/2505] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:57:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:57:21.341507 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0fe64-72eb-473f-a54c-8478e8f18352 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:21.343048 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-73d0fe64-72eb-473f-a54c-8478e8f18352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:21.358627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0fe64-72eb-473f-a54c-8478e8f18352 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:21.365452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40864e74-3b61-4180-9d36-8c9ff4f12da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:21.366011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40864e74-3b61-4180-9d36-8c9ff4f12da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:21.366297 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40864e74-3b61-4180-9d36-8c9ff4f12da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:21.366579 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40864e74-3b61-4180-9d36-8c9ff4f12da8 admin admin] PATH_INFO: `/v3/projects/1d8627531fef472faaf15229336c49de` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:21.367833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40864e74-3b61-4180-9d36-8c9ff4f12da8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d8627531fef472faaf15229336c49de)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:21.369758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40864e74-3b61-4180-9d36-8c9ff4f12da8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:21.505491 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-40864e74-3b61-4180-9d36-8c9ff4f12da8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d8627531fef472faaf15229336c49de', 'request_id': 'req-40864e74-3b61-4180-9d36-8c9ff4f12da8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:21.512137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-40864e74-3b61-4180-9d36-8c9ff4f12da8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d8627531fef472faaf15229336c49de', 'request_id': 'req-40864e74-3b61-4180-9d36-8c9ff4f12da8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:21.518096 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-40864e74-3b61-4180-9d36-8c9ff4f12da8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d8627531fef472faaf15229336c49de', 'request_id': 'req-40864e74-3b61-4180-9d36-8c9ff4f12da8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:21.602569 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-40864e74-3b61-4180-9d36-8c9ff4f12da8 admin admin] The token cache is being invalidate because project 1d8627531fef472faaf15229336c49de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:57:21.604671 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-40864e74-3b61-4180-9d36-8c9ff4f12da8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:21.604671 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-40864e74-3b61-4180-9d36-8c9ff4f12da8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:21.605249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1266/2506] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:57:21 2022] DELETE /identity/v3/projects/1d8627531fef472faaf15229336c49de => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:57:21.609790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f04f8a-fb5e-47b3-8958-3f82aed9314c None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:21.610253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f04f8a-fb5e-47b3-8958-3f82aed9314c None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:21.610372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f04f8a-fb5e-47b3-8958-3f82aed9314c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:21.631696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-22f04f8a-fb5e-47b3-8958-3f82aed9314c None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:21.632928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-22f04f8a-fb5e-47b3-8958-3f82aed9314c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:21.650739 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:21.695605 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1241/2507] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:57:21.703148 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df98ef-ce06-4c5b-8b8d-e6d247696d1c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:21.703460 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df98ef-ce06-4c5b-8b8d-e6d247696d1c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:21.703635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df98ef-ce06-4c5b-8b8d-e6d247696d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:21.724771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f5df98ef-ce06-4c5b-8b8d-e6d247696d1c None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:21.726085 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f5df98ef-ce06-4c5b-8b8d-e6d247696d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:21.734353 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1267/2508] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:57:21.738971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f04f8a-fb5e-47b3-8958-3f82aed9314c None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:21.750013 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f04f8a-fb5e-47b3-8958-3f82aed9314c None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:21.756424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8161c4f-4f70-4f13-a387-9ec44e892499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:21.757056 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8161c4f-4f70-4f13-a387-9ec44e892499 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:21.757340 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8161c4f-4f70-4f13-a387-9ec44e892499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:21.757614 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8161c4f-4f70-4f13-a387-9ec44e892499 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:21.758272 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8161c4f-4f70-4f13-a387-9ec44e892499 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:57:21.758575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8161c4f-4f70-4f13-a387-9ec44e892499 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:21.760354 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8161c4f-4f70-4f13-a387-9ec44e892499 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:21.905252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1242/2509] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:57:21.905864 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df98ef-ce06-4c5b-8b8d-e6d247696d1c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:21.905864 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df98ef-ce06-4c5b-8b8d-e6d247696d1c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:21.905864 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467e860-69d3-44f9-a614-db0b4ee4f563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:21.905864 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467e860-69d3-44f9-a614-db0b4ee4f563 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:21.906334 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467e860-69d3-44f9-a614-db0b4ee4f563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:21.906334 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467e860-69d3-44f9-a614-db0b4ee4f563 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:21.906334 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467e860-69d3-44f9-a614-db0b4ee4f563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:21.906334 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467e860-69d3-44f9-a614-db0b4ee4f563 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:21.984676 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1268/2510] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:57:21 2022] POST /identity/v3/projects => generated 373 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:57:21.986311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8161c4f-4f70-4f13-a387-9ec44e892499 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:21.996622 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8161c4f-4f70-4f13-a387-9ec44e892499 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:22.053354 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a8ad5-0bba-4e57-8fb7-07e853cc8235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:22.053965 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a8ad5-0bba-4e57-8fb7-07e853cc8235 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:22.054246 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a8ad5-0bba-4e57-8fb7-07e853cc8235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:22.054521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a8ad5-0bba-4e57-8fb7-07e853cc8235 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:22.055149 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a8ad5-0bba-4e57-8fb7-07e853cc8235 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:22.057129 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a8ad5-0bba-4e57-8fb7-07e853cc8235 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:22.212871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1243/2511] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:57:21 2022] POST /identity/v3/users => generated 287 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:57:22.217056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467e860-69d3-44f9-a614-db0b4ee4f563 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:22.227129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467e860-69d3-44f9-a614-db0b4ee4f563 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:22.231878 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7154a-34f4-4043-9335-bb639bfa239b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:22.232514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7154a-34f4-4043-9335-bb639bfa239b admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:22.232798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7154a-34f4-4043-9335-bb639bfa239b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:22.233072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7154a-34f4-4043-9335-bb639bfa239b admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:22.233717 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7154a-34f4-4043-9335-bb639bfa239b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:22.235329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7154a-34f4-4043-9335-bb639bfa239b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:22.246112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1269/2512] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:57:22.251274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a8ad5-0bba-4e57-8fb7-07e853cc8235 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:22.262134 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a8ad5-0bba-4e57-8fb7-07e853cc8235 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:22.266743 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5214e99-d523-4e42-9ae5-123cbd5f760c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:22.267341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5214e99-d523-4e42-9ae5-123cbd5f760c admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:22.267621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5214e99-d523-4e42-9ae5-123cbd5f760c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:22.267893 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5214e99-d523-4e42-9ae5-123cbd5f760c admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:22.268578 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5214e99-d523-4e42-9ae5-123cbd5f760c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:22.270185 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5214e99-d523-4e42-9ae5-123cbd5f760c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:22.281292 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1244/2513] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:57:22.332843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7154a-34f4-4043-9335-bb639bfa239b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:22.332843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7154a-34f4-4043-9335-bb639bfa239b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:22.332843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09096b63-cbe3-4b00-bbad-796fad30a8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:22.332843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09096b63-cbe3-4b00-bbad-796fad30a8b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:22.333324 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09096b63-cbe3-4b00-bbad-796fad30a8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:22.333324 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09096b63-cbe3-4b00-bbad-796fad30a8b1 admin admin] PATH_INFO: `/v3/projects/9ebba8d99c8847ddb971578fb19cd6e2/users/2c77c2b9101541da83f67b69990fa205/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:22.333324 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09096b63-cbe3-4b00-bbad-796fad30a8b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ebba8d99c8847ddb971578fb19cd6e2, user_id=2c77c2b9101541da83f67b69990fa205, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:22.333582 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09096b63-cbe3-4b00-bbad-796fad30a8b1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:22.414385 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1270/2514] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:57:22 2022] PUT /identity/v3/projects/9ebba8d99c8847ddb971578fb19cd6e2/users/2c77c2b9101541da83f67b69990fa205/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:57:22.424689 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e27d6d-a951-4174-8670-2d0821ec48df None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:22.424977 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e27d6d-a951-4174-8670-2d0821ec48df None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:22.425276 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e27d6d-a951-4174-8670-2d0821ec48df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:22.454176 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-72e27d6d-a951-4174-8670-2d0821ec48df None None] MFA Rules not processed for user `2c77c2b9101541da83f67b69990fa205`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:22.455392 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-72e27d6d-a951-4174-8670-2d0821ec48df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:22.518625 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1245/2515] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:57:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:57:22.529602 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09096b63-cbe3-4b00-bbad-796fad30a8b1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:22.531304 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-09096b63-cbe3-4b00-bbad-796fad30a8b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:22.554996 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:22.589384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09096b63-cbe3-4b00-bbad-796fad30a8b1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:22.593081 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8998f8-14a1-4be4-9596-9f60d4bb2786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:22.593695 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8998f8-14a1-4be4-9596-9f60d4bb2786 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:22.593972 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8998f8-14a1-4be4-9596-9f60d4bb2786 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:22.594244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8998f8-14a1-4be4-9596-9f60d4bb2786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:22.603285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8998f8-14a1-4be4-9596-9f60d4bb2786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:22.604768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8998f8-14a1-4be4-9596-9f60d4bb2786 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:22.623079 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1271/2516] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:57:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:57:24.334220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3569-92f5-49f4-8cbd-0e18d824f87a None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:24.340825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3569-92f5-49f4-8cbd-0e18d824f87a None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:24.340825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3569-92f5-49f4-8cbd-0e18d824f87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:24.353668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-cfec3569-92f5-49f4-8cbd-0e18d824f87a None None] MFA Rules not processed for user `2c77c2b9101541da83f67b69990fa205`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:24.354864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-cfec3569-92f5-49f4-8cbd-0e18d824f87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:24.361776 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1246/2517] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:57:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:57:33.373689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=17 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:34.348430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:39.194247 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b011e-e1e2-442c-9497-fdfcc19b6ad0 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:39.194742 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b011e-e1e2-442c-9497-fdfcc19b6ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:39.194853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b011e-e1e2-442c-9497-fdfcc19b6ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:39.225366 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-968b011e-e1e2-442c-9497-fdfcc19b6ad0 None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:39.226573 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-968b011e-e1e2-442c-9497-fdfcc19b6ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:39.288946 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1272/2518] 10.222.0.70 () {58 vars in 1009 bytes} [Wed Aug 24 16:57:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:57:39.430241 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec3569-92f5-49f4-8cbd-0e18d824f87a None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:39.431690 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-cfec3569-92f5-49f4-8cbd-0e18d824f87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:39.511425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec3569-92f5-49f4-8cbd-0e18d824f87a None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:39.516468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea1ed6-c375-4f2c-9de1-182d20346971 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:39.517240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea1ed6-c375-4f2c-9de1-182d20346971 service swift] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:39.517524 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea1ed6-c375-4f2c-9de1-182d20346971 service swift] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:39.517797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea1ed6-c375-4f2c-9de1-182d20346971 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:39.526264 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea1ed6-c375-4f2c-9de1-182d20346971 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:39.527865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea1ed6-c375-4f2c-9de1-182d20346971 service swift] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:39.536626 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1247/2519] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 16:57:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:57:43.447376 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98265307-458d-4a80-80b6-a59d5319be6f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:43.447908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98265307-458d-4a80-80b6-a59d5319be6f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:43.448173 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98265307-458d-4a80-80b6-a59d5319be6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:43.482222 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-98265307-458d-4a80-80b6-a59d5319be6f None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:43.483730 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-98265307-458d-4a80-80b6-a59d5319be6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:43.501691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1273/2520] 10.222.0.70 () {58 vars in 1009 bytes} [Wed Aug 24 16:57:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:57:43.518253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea1ed6-c375-4f2c-9de1-182d20346971 service swift] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:43.531893 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea1ed6-c375-4f2c-9de1-182d20346971 service swift] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:43.542657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9deea-c9d6-4e6a-9468-1178ccd901c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:43.543284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9deea-c9d6-4e6a-9468-1178ccd901c5 service swift] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:43.543569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9deea-c9d6-4e6a-9468-1178ccd901c5 service swift] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:43.543845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9deea-c9d6-4e6a-9468-1178ccd901c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:43.555349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9deea-c9d6-4e6a-9468-1178ccd901c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:43.556683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9deea-c9d6-4e6a-9468-1178ccd901c5 service swift] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:43.570530 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1248/2521] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 16:57:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:57:44.695397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346bda8-0f53-48c4-aef6-0921276f1b06 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:44.695921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346bda8-0f53-48c4-aef6-0921276f1b06 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:44.696157 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346bda8-0f53-48c4-aef6-0921276f1b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:44.724316 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4346bda8-0f53-48c4-aef6-0921276f1b06 None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:44.724316 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4346bda8-0f53-48c4-aef6-0921276f1b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:44.730561 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1274/2522] 10.222.0.70 () {58 vars in 967 bytes} [Wed Aug 24 16:57:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:57:44.747090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9deea-c9d6-4e6a-9468-1178ccd901c5 service swift] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:44.758168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9deea-c9d6-4e6a-9468-1178ccd901c5 service swift] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:44.764182 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db079f24-ea4b-4ff1-9a6c-f9d2f3451abc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:44.764833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db079f24-ea4b-4ff1-9a6c-f9d2f3451abc service swift] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:44.765304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db079f24-ea4b-4ff1-9a6c-f9d2f3451abc service swift] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:44.765737 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db079f24-ea4b-4ff1-9a6c-f9d2f3451abc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:44.775356 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db079f24-ea4b-4ff1-9a6c-f9d2f3451abc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:44.777305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db079f24-ea4b-4ff1-9a6c-f9d2f3451abc service swift] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:44.786915 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1249/2523] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 16:57:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:57:45.311244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346bda8-0f53-48c4-aef6-0921276f1b06 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:45.312696 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4346bda8-0f53-48c4-aef6-0921276f1b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:45.371301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346bda8-0f53-48c4-aef6-0921276f1b06 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:45.374910 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a4032-194e-422f-97cd-672dd0fa7ed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:45.375548 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a4032-194e-422f-97cd-672dd0fa7ed1 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:45.375832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a4032-194e-422f-97cd-672dd0fa7ed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:45.376109 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a4032-194e-422f-97cd-672dd0fa7ed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:45.377879 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0d7a4032-194e-422f-97cd-672dd0fa7ed1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:45.439644 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a4032-194e-422f-97cd-672dd0fa7ed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:45.441201 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a4032-194e-422f-97cd-672dd0fa7ed1 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:45.451262 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1275/2524] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:57:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:57:46.054180 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91831f-d37b-402b-8286-a36b4a70134d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:46.055043 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91831f-d37b-402b-8286-a36b4a70134d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:46.055480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91831f-d37b-402b-8286-a36b4a70134d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:46.080586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-ab91831f-d37b-402b-8286-a36b4a70134d None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:46.081824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ab91831f-d37b-402b-8286-a36b4a70134d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:46.090680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1250/2525] 10.222.0.70 () {58 vars in 1009 bytes} [Wed Aug 24 16:57:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:57:46.122180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a4032-194e-422f-97cd-672dd0fa7ed1 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:46.138903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a4032-194e-422f-97cd-672dd0fa7ed1 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:46.145308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3efc5-776f-4b97-bf23-6014bc248ce9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:46.146281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3efc5-776f-4b97-bf23-6014bc248ce9 service swift] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:46.146754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3efc5-776f-4b97-bf23-6014bc248ce9 service swift] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:46.147218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3efc5-776f-4b97-bf23-6014bc248ce9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:46.160247 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3efc5-776f-4b97-bf23-6014bc248ce9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:46.162231 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3efc5-776f-4b97-bf23-6014bc248ce9 service swift] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:46.174586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1276/2526] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 16:57:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:57:46.939756 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398a582-5c75-462e-bd7d-82ea98ad4589 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:46.940446 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398a582-5c75-462e-bd7d-82ea98ad4589 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:46.941063 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398a582-5c75-462e-bd7d-82ea98ad4589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:46.961959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-1398a582-5c75-462e-bd7d-82ea98ad4589 None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:46.963110 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1398a582-5c75-462e-bd7d-82ea98ad4589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:47.003604 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1251/2527] 10.222.0.70 () {58 vars in 967 bytes} [Wed Aug 24 16:57:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:57:47.017003 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3efc5-776f-4b97-bf23-6014bc248ce9 service swift] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:47.033541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3efc5-776f-4b97-bf23-6014bc248ce9 service swift] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:47.041767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a776b7-3f82-48a9-9c3e-100efb5154eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:47.042447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a776b7-3f82-48a9-9c3e-100efb5154eb service swift] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:47.042665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a776b7-3f82-48a9-9c3e-100efb5154eb service swift] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:47.043658 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a776b7-3f82-48a9-9c3e-100efb5154eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:47.057874 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a776b7-3f82-48a9-9c3e-100efb5154eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:47.060994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a776b7-3f82-48a9-9c3e-100efb5154eb service swift] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:47.077439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1277/2528] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 16:57:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:57:51.058501 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398a582-5c75-462e-bd7d-82ea98ad4589 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:51.127507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398a582-5c75-462e-bd7d-82ea98ad4589 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:51.132644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09c10a-fa6d-4350-bdae-0fea6b8c5813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:51.133281 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09c10a-fa6d-4350-bdae-0fea6b8c5813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:51.133569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09c10a-fa6d-4350-bdae-0fea6b8c5813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:51.134343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09c10a-fa6d-4350-bdae-0fea6b8c5813 admin admin] PATH_INFO: `/v3/users/2c77c2b9101541da83f67b69990fa205` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:51.135496 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09c10a-fa6d-4350-bdae-0fea6b8c5813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c77c2b9101541da83f67b69990fa205)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:51.137528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09c10a-fa6d-4350-bdae-0fea6b8c5813 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:51.422732 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ec09c10a-fa6d-4350-bdae-0fea6b8c5813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c77c2b9101541da83f67b69990fa205'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:51.424026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ec09c10a-fa6d-4350-bdae-0fea6b8c5813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c77c2b9101541da83f67b69990fa205'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:51.424026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ec09c10a-fa6d-4350-bdae-0fea6b8c5813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c77c2b9101541da83f67b69990fa205'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:51.429879 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:51.432477 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ec09c10a-fa6d-4350-bdae-0fea6b8c5813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c77c2b9101541da83f67b69990fa205'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:51.546246 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ec09c10a-fa6d-4350-bdae-0fea6b8c5813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c77c2b9101541da83f67b69990fa205'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:51.564663 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1252/2529] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:57:51 2022] DELETE /identity/v3/users/2c77c2b9101541da83f67b69990fa205 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:57:51.569445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0806ab-5011-47be-8700-0a9ca6c492ff None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:51.569900 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0806ab-5011-47be-8700-0a9ca6c492ff None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:51.570146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0806ab-5011-47be-8700-0a9ca6c492ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:51.578411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:51.589717 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3f0806ab-5011-47be-8700-0a9ca6c492ff None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:51.590919 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3f0806ab-5011-47be-8700-0a9ca6c492ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:51.648316 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1278/2530] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:57:51.658347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09c10a-fa6d-4350-bdae-0fea6b8c5813 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:51.659727 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ec09c10a-fa6d-4350-bdae-0fea6b8c5813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:51.718393 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09c10a-fa6d-4350-bdae-0fea6b8c5813 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:51.721997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba821d-2c02-4a5e-bbc2-bd64d023f794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:51.722580 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba821d-2c02-4a5e-bbc2-bd64d023f794 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:51.722859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba821d-2c02-4a5e-bbc2-bd64d023f794 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:51.723131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba821d-2c02-4a5e-bbc2-bd64d023f794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:51.732559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba821d-2c02-4a5e-bbc2-bd64d023f794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:51.733969 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba821d-2c02-4a5e-bbc2-bd64d023f794 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:51.743287 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1253/2531] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:57:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:57:51.993144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0806ab-5011-47be-8700-0a9ca6c492ff None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:51.994625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3f0806ab-5011-47be-8700-0a9ca6c492ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:52.007557 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0806ab-5011-47be-8700-0a9ca6c492ff None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:52.012536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3e3fb-05c9-4e86-b6e9-2c959c4b51ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:52.013382 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3e3fb-05c9-4e86-b6e9-2c959c4b51ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:52.013490 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3e3fb-05c9-4e86-b6e9-2c959c4b51ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:52.013737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3e3fb-05c9-4e86-b6e9-2c959c4b51ae admin admin] PATH_INFO: `/v3/projects/9ebba8d99c8847ddb971578fb19cd6e2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:52.014961 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3e3fb-05c9-4e86-b6e9-2c959c4b51ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ebba8d99c8847ddb971578fb19cd6e2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:52.016948 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3e3fb-05c9-4e86-b6e9-2c959c4b51ae admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:52.141001 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-16f3e3fb-05c9-4e86-b6e9-2c959c4b51ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ebba8d99c8847ddb971578fb19cd6e2', 'request_id': 'req-16f3e3fb-05c9-4e86-b6e9-2c959c4b51ae'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:52.148757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-16f3e3fb-05c9-4e86-b6e9-2c959c4b51ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ebba8d99c8847ddb971578fb19cd6e2', 'request_id': 'req-16f3e3fb-05c9-4e86-b6e9-2c959c4b51ae'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:52.156495 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-16f3e3fb-05c9-4e86-b6e9-2c959c4b51ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ebba8d99c8847ddb971578fb19cd6e2', 'request_id': 'req-16f3e3fb-05c9-4e86-b6e9-2c959c4b51ae'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:52.215249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-16f3e3fb-05c9-4e86-b6e9-2c959c4b51ae admin admin] The token cache is being invalidate because project 9ebba8d99c8847ddb971578fb19cd6e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:57:52.215829 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-16f3e3fb-05c9-4e86-b6e9-2c959c4b51ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:52.216935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-16f3e3fb-05c9-4e86-b6e9-2c959c4b51ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:52.218860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1279/2532] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:57:51 2022] DELETE /identity/v3/projects/9ebba8d99c8847ddb971578fb19cd6e2 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:57:52.223942 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad363d-6b43-4e97-a063-4f293ad21b32 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:52.225128 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad363d-6b43-4e97-a063-4f293ad21b32 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:52.225128 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad363d-6b43-4e97-a063-4f293ad21b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:52.248341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-b0ad363d-6b43-4e97-a063-4f293ad21b32 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:52.249726 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b0ad363d-6b43-4e97-a063-4f293ad21b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:52.323833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1254/2533] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:57:52.331556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef3de4-be4d-464a-8f08-cfb9163426a9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:52.331843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef3de4-be4d-464a-8f08-cfb9163426a9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:52.332284 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef3de4-be4d-464a-8f08-cfb9163426a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:52.352680 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-abef3de4-be4d-464a-8f08-cfb9163426a9 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:52.353996 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-abef3de4-be4d-464a-8f08-cfb9163426a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:52.361924 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1280/2534] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:57:52.366113 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad363d-6b43-4e97-a063-4f293ad21b32 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:52.376147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad363d-6b43-4e97-a063-4f293ad21b32 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:52.381290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e758716-d4be-49e5-a99b-57094614ca2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:52.381903 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e758716-d4be-49e5-a99b-57094614ca2f admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:52.382187 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e758716-d4be-49e5-a99b-57094614ca2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:52.382468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e758716-d4be-49e5-a99b-57094614ca2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:52.383119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e758716-d4be-49e5-a99b-57094614ca2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:57:52.383424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e758716-d4be-49e5-a99b-57094614ca2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:52.385663 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e758716-d4be-49e5-a99b-57094614ca2f admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:52.402124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1255/2535] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:57:52.403263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef3de4-be4d-464a-8f08-cfb9163426a9 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:52.412559 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef3de4-be4d-464a-8f08-cfb9163426a9 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:52.417736 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938c1c6-58f3-415e-a10f-3e30cfceeeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:52.418354 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938c1c6-58f3-415e-a10f-3e30cfceeeaf admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:52.422631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938c1c6-58f3-415e-a10f-3e30cfceeeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:52.422631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938c1c6-58f3-415e-a10f-3e30cfceeeaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:52.422631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938c1c6-58f3-415e-a10f-3e30cfceeeaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:52.422631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938c1c6-58f3-415e-a10f-3e30cfceeeaf admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:52.548200 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e758716-d4be-49e5-a99b-57094614ca2f admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:52.550008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-8e758716-d4be-49e5-a99b-57094614ca2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:52.553329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1281/2536] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:57:52 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:57:52.557399 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938c1c6-58f3-415e-a10f-3e30cfceeeaf admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:52.608643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e758716-d4be-49e5-a99b-57094614ca2f admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:52.617903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:52.620455 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:52.629099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938c1c6-58f3-415e-a10f-3e30cfceeeaf admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:52.634712 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec84ae-caad-47cf-a701-fe3676bd7f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:52.635439 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec84ae-caad-47cf-a701-fe3676bd7f3a admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:52.635736 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec84ae-caad-47cf-a701-fe3676bd7f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:52.636011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec84ae-caad-47cf-a701-fe3676bd7f3a admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:52.636678 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec84ae-caad-47cf-a701-fe3676bd7f3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:52.638609 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec84ae-caad-47cf-a701-fe3676bd7f3a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:52.658556 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b436c5-018e-4ea3-8866-f75d8a337bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:52.659168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b436c5-018e-4ea3-8866-f75d8a337bdf service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:52.659457 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b436c5-018e-4ea3-8866-f75d8a337bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:52.659714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b436c5-018e-4ea3-8866-f75d8a337bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:52.661475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-98b436c5-018e-4ea3-8866-f75d8a337bdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:52.673124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b436c5-018e-4ea3-8866-f75d8a337bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:52.674519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b436c5-018e-4ea3-8866-f75d8a337bdf service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:52.684514 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1256/2537] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:57:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:57:52.743690 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1282/2538] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:57:52 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:57:52.744845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b436c5-018e-4ea3-8866-f75d8a337bdf service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:52.754570 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b436c5-018e-4ea3-8866-f75d8a337bdf service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:52.760683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89450fa4-b0dc-4ecc-97c5-6b8ddbbcb819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:52.761277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89450fa4-b0dc-4ecc-97c5-6b8ddbbcb819 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:52.761555 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89450fa4-b0dc-4ecc-97c5-6b8ddbbcb819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:52.764126 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89450fa4-b0dc-4ecc-97c5-6b8ddbbcb819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:52.764126 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89450fa4-b0dc-4ecc-97c5-6b8ddbbcb819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:52.764126 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89450fa4-b0dc-4ecc-97c5-6b8ddbbcb819 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:52.774974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1257/2539] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:57:52.778892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec84ae-caad-47cf-a701-fe3676bd7f3a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:52.789089 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec84ae-caad-47cf-a701-fe3676bd7f3a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:52.794934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9fefb5-f8de-4567-8938-2a7300486d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:52.795551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fefb5-f8de-4567-8938-2a7300486d9e admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:52.795954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fefb5-f8de-4567-8938-2a7300486d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:52.796238 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fefb5-f8de-4567-8938-2a7300486d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:52.796942 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9fefb5-f8de-4567-8938-2a7300486d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:52.798546 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9fefb5-f8de-4567-8938-2a7300486d9e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:52.817309 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1283/2540] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:57:52.822277 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89450fa4-b0dc-4ecc-97c5-6b8ddbbcb819 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:52.832490 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89450fa4-b0dc-4ecc-97c5-6b8ddbbcb819 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:52.837360 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282a818-9191-4725-84ca-7134f3b8c496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:52.838016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282a818-9191-4725-84ca-7134f3b8c496 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:52.838299 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282a818-9191-4725-84ca-7134f3b8c496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:52.838601 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282a818-9191-4725-84ca-7134f3b8c496 admin admin] PATH_INFO: `/v3/projects/1088b1d454684ca5be800a99d4e26304/users/d715cc7828f847529f795f356a8ac16b/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:52.854480 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282a818-9191-4725-84ca-7134f3b8c496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1088b1d454684ca5be800a99d4e26304, user_id=d715cc7828f847529f795f356a8ac16b, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:52.857205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282a818-9191-4725-84ca-7134f3b8c496 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:52.935079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1258/2541] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:57:52 2022] PUT /identity/v3/projects/1088b1d454684ca5be800a99d4e26304/users/d715cc7828f847529f795f356a8ac16b/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:57:52.940000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15f8cb-2c13-4874-9010-c21e91e1a33f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:52.940287 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15f8cb-2c13-4874-9010-c21e91e1a33f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:52.940599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15f8cb-2c13-4874-9010-c21e91e1a33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:52.959723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7b15f8cb-2c13-4874-9010-c21e91e1a33f None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:52.960952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7b15f8cb-2c13-4874-9010-c21e91e1a33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:53.031092 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1284/2542] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:57:53.041285 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282a818-9191-4725-84ca-7134f3b8c496 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:53.042976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e282a818-9191-4725-84ca-7134f3b8c496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:53.100777 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282a818-9191-4725-84ca-7134f3b8c496 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:53.105081 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7972e99-fc2d-44ae-9295-76ec167fc5cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:53.105689 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7972e99-fc2d-44ae-9295-76ec167fc5cf service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:53.105964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7972e99-fc2d-44ae-9295-76ec167fc5cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:53.106233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7972e99-fc2d-44ae-9295-76ec167fc5cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:53.115702 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7972e99-fc2d-44ae-9295-76ec167fc5cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:53.117323 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7972e99-fc2d-44ae-9295-76ec167fc5cf service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:53.127407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1259/2543] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:57:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:57:54.539737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c594f21-64f6-4740-8934-81765dd77489 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:54.540869 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c594f21-64f6-4740-8934-81765dd77489 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:54.540869 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c594f21-64f6-4740-8934-81765dd77489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:54.595243 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7c594f21-64f6-4740-8934-81765dd77489 None None] MFA Rules not processed for user `d715cc7828f847529f795f356a8ac16b`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:54.596674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7c594f21-64f6-4740-8934-81765dd77489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:54.681408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1285/2544] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:57:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:57:54.692874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7972e99-fc2d-44ae-9295-76ec167fc5cf service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:54.694422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b7972e99-fc2d-44ae-9295-76ec167fc5cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:54.762144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7972e99-fc2d-44ae-9295-76ec167fc5cf service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:54.768005 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0ad86-671a-4f64-bef4-23c6ab8c57ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:54.768463 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0ad86-671a-4f64-bef4-23c6ab8c57ef service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:54.768759 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0ad86-671a-4f64-bef4-23c6ab8c57ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:54.769040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0ad86-671a-4f64-bef4-23c6ab8c57ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:54.778832 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d0ad86-671a-4f64-bef4-23c6ab8c57ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:54.780615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d0ad86-671a-4f64-bef4-23c6ab8c57ef service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:54.791306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1260/2545] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:57:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:57:55.665041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c594f21-64f6-4740-8934-81765dd77489 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:55.665920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c594f21-64f6-4740-8934-81765dd77489 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:55.675826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1151e11-3cb3-44ff-9abd-c56c3d664fa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:55.676478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1151e11-3cb3-44ff-9abd-c56c3d664fa9 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:55.676766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1151e11-3cb3-44ff-9abd-c56c3d664fa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:55.677082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1151e11-3cb3-44ff-9abd-c56c3d664fa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:55.692852 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e1151e11-3cb3-44ff-9abd-c56c3d664fa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:55.777091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:55.816329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1151e11-3cb3-44ff-9abd-c56c3d664fa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:55.817826 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1151e11-3cb3-44ff-9abd-c56c3d664fa9 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:55.839169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1286/2546] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:57:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:57:56.568620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0ad86-671a-4f64-bef4-23c6ab8c57ef service cinder] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:56.570640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-47d0ad86-671a-4f64-bef4-23c6ab8c57ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:56.611112 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:56.628113 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8990b-76d8-40bb-94df-701f204e9a40 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:56.628425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8990b-76d8-40bb-94df-701f204e9a40 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:56.628741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8990b-76d8-40bb-94df-701f204e9a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:56.648802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c7d8990b-76d8-40bb-94df-701f204e9a40 None None] MFA Rules not processed for user `d715cc7828f847529f795f356a8ac16b`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:56.650045 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c7d8990b-76d8-40bb-94df-701f204e9a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:56.653104 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0ad86-671a-4f64-bef4-23c6ab8c57ef service cinder] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:56.657120 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1287/2547] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:57:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:57:56.658238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddbfe6-083c-4cf9-b288-b2ec727e2847 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:56.658839 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddbfe6-083c-4cf9-b288-b2ec727e2847 service placement] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:56.659119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddbfe6-083c-4cf9-b288-b2ec727e2847 service placement] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:56.659389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddbfe6-083c-4cf9-b288-b2ec727e2847 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:56.661266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-f9ddbfe6-083c-4cf9-b288-b2ec727e2847 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:56.673195 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ddbfe6-083c-4cf9-b288-b2ec727e2847 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:56.675001 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ddbfe6-083c-4cf9-b288-b2ec727e2847 service placement] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:56.687629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1261/2548] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:57:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:57:57.532499 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8990b-76d8-40bb-94df-701f204e9a40 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:57.543760 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8990b-76d8-40bb-94df-701f204e9a40 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:57.548913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660d89e-b1ab-433c-8b97-0fc0c204a88e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:57.549902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660d89e-b1ab-433c-8b97-0fc0c204a88e service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:57.550250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660d89e-b1ab-433c-8b97-0fc0c204a88e service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:57.550588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660d89e-b1ab-433c-8b97-0fc0c204a88e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:57.552690 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7660d89e-b1ab-433c-8b97-0fc0c204a88e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:57.587914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660d89e-b1ab-433c-8b97-0fc0c204a88e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:57.590780 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660d89e-b1ab-433c-8b97-0fc0c204a88e service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:57.603540 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1288/2549] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:57:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:57:58.635650 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e1d50-ff87-4479-9816-f0dc4c3d40b3 None None] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:58.637256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e1d50-ff87-4479-9816-f0dc4c3d40b3 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:58.637765 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e1d50-ff87-4479-9816-f0dc4c3d40b3 None None] PATH_INFO: `/` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:58.640331 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1262/2550] 10.222.0.70 () {54 vars in 921 bytes} [Wed Aug 24 16:57:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:57:58.646861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813be040-6726-4740-96b2-3797744447a3 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:58.647703 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813be040-6726-4740-96b2-3797744447a3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:58.648187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813be040-6726-4740-96b2-3797744447a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:58.700281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-813be040-6726-4740-96b2-3797744447a3 None None] MFA Rules not processed for user `e333fca7790244f6a4ebceabb55faba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:58.702490 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-813be040-6726-4740-96b2-3797744447a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:58.717612 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1289/2551] 10.222.0.70 () {58 vars in 1018 bytes} [Wed Aug 24 16:57:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:57:58.731596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e1d50-ff87-4479-9816-f0dc4c3d40b3 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:58.733676 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-783e1d50-ff87-4479-9816-f0dc4c3d40b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:58.796333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e1d50-ff87-4479-9816-f0dc4c3d40b3 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:58.800448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e500b4d-007c-4dab-8566-0dc4f950ad6b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56418ff737504181a59714aad9f1c375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:58.801064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e500b4d-007c-4dab-8566-0dc4f950ad6b service glance] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:58.801343 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e500b4d-007c-4dab-8566-0dc4f950ad6b service glance] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:58.801689 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e500b4d-007c-4dab-8566-0dc4f950ad6b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:58.813316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e500b4d-007c-4dab-8566-0dc4f950ad6b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:58.814748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e500b4d-007c-4dab-8566-0dc4f950ad6b service glance] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:58.826585 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1263/2552] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:57:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:58:01.205841 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813be040-6726-4740-96b2-3797744447a3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:01.208081 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-813be040-6726-4740-96b2-3797744447a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:01.227313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813be040-6726-4740-96b2-3797744447a3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:01.234054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48202b9-6a57-4cd1-a07d-43a5f945f24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:01.234923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48202b9-6a57-4cd1-a07d-43a5f945f24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:01.235626 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48202b9-6a57-4cd1-a07d-43a5f945f24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:01.236772 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48202b9-6a57-4cd1-a07d-43a5f945f24f admin admin] PATH_INFO: `/v3/users/90e3f9b6511347a692bed976865d5fad` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:01.239244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48202b9-6a57-4cd1-a07d-43a5f945f24f admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e3f9b6511347a692bed976865d5fad)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:01.241658 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48202b9-6a57-4cd1-a07d-43a5f945f24f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:01.472609 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a48202b9-6a57-4cd1-a07d-43a5f945f24f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e3f9b6511347a692bed976865d5fad'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:01.473570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a48202b9-6a57-4cd1-a07d-43a5f945f24f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e3f9b6511347a692bed976865d5fad'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:01.474263 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a48202b9-6a57-4cd1-a07d-43a5f945f24f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e3f9b6511347a692bed976865d5fad'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:01.585635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a48202b9-6a57-4cd1-a07d-43a5f945f24f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e3f9b6511347a692bed976865d5fad'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:01.598653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a48202b9-6a57-4cd1-a07d-43a5f945f24f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e3f9b6511347a692bed976865d5fad'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:01.618529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1290/2553] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:58:01 2022] DELETE /identity/v3/users/90e3f9b6511347a692bed976865d5fad => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:58:01.890239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e500b4d-007c-4dab-8566-0dc4f950ad6b service glance] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:01.902231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e500b4d-007c-4dab-8566-0dc4f950ad6b service glance] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:01.980976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18ae95-f67f-44bb-9761-b322df76c0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:01.981534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18ae95-f67f-44bb-9761-b322df76c0b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:01.981813 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18ae95-f67f-44bb-9761-b322df76c0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:01.982088 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18ae95-f67f-44bb-9761-b322df76c0b5 admin admin] PATH_INFO: `/v3/projects/1715333032634911bcf0f7a41fab587a` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:01.983206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18ae95-f67f-44bb-9761-b322df76c0b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1715333032634911bcf0f7a41fab587a)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:01.985157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18ae95-f67f-44bb-9761-b322df76c0b5 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:02.095243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-8b18ae95-f67f-44bb-9761-b322df76c0b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1715333032634911bcf0f7a41fab587a', 'request_id': 'req-8b18ae95-f67f-44bb-9761-b322df76c0b5'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:02.103458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-8b18ae95-f67f-44bb-9761-b322df76c0b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1715333032634911bcf0f7a41fab587a', 'request_id': 'req-8b18ae95-f67f-44bb-9761-b322df76c0b5'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:02.109433 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-8b18ae95-f67f-44bb-9761-b322df76c0b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1715333032634911bcf0f7a41fab587a', 'request_id': 'req-8b18ae95-f67f-44bb-9761-b322df76c0b5'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:02.111216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:58:02.185113 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-8b18ae95-f67f-44bb-9761-b322df76c0b5 admin admin] The token cache is being invalidate because project 1715333032634911bcf0f7a41fab587a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:58:02.185403 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-8b18ae95-f67f-44bb-9761-b322df76c0b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:02.186039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-8b18ae95-f67f-44bb-9761-b322df76c0b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:02.187618 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1264/2554] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:58:01 2022] DELETE /identity/v3/projects/1715333032634911bcf0f7a41fab587a => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:58:02.191750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48202b9-6a57-4cd1-a07d-43a5f945f24f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:02.193207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a48202b9-6a57-4cd1-a07d-43a5f945f24f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:02.206692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48202b9-6a57-4cd1-a07d-43a5f945f24f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:02.257692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:58:02.263950 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8893a6-fea3-4fad-bb95-39018feb7b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:02.264627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8893a6-fea3-4fad-bb95-39018feb7b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:02.264914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8893a6-fea3-4fad-bb95-39018feb7b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:02.265192 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8893a6-fea3-4fad-bb95-39018feb7b50 admin admin] PATH_INFO: `/v3/users/40426d2d806f4648a107abb2393b8d01` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:02.266339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8893a6-fea3-4fad-bb95-39018feb7b50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40426d2d806f4648a107abb2393b8d01)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:02.271371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8893a6-fea3-4fad-bb95-39018feb7b50 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:02.465074 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee3b31-1f55-4cb2-acb3-5f95d73e56a7 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:02.465429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee3b31-1f55-4cb2-acb3-5f95d73e56a7 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:02.465773 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee3b31-1f55-4cb2-acb3-5f95d73e56a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:02.474171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3d8893a6-fea3-4fad-bb95-39018feb7b50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40426d2d806f4648a107abb2393b8d01'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:02.474905 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3d8893a6-fea3-4fad-bb95-39018feb7b50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40426d2d806f4648a107abb2393b8d01'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:02.475728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3d8893a6-fea3-4fad-bb95-39018feb7b50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40426d2d806f4648a107abb2393b8d01'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:02.488667 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-dcee3b31-1f55-4cb2-acb3-5f95d73e56a7 None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:58:02.489928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-dcee3b31-1f55-4cb2-acb3-5f95d73e56a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:02.553059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1265/2555] 10.222.0.70 () {58 vars in 1009 bytes} [Wed Aug 24 16:58:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:58:02.566613 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3d8893a6-fea3-4fad-bb95-39018feb7b50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40426d2d806f4648a107abb2393b8d01'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:02.567507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee3b31-1f55-4cb2-acb3-5f95d73e56a7 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:02.570543 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-dcee3b31-1f55-4cb2-acb3-5f95d73e56a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:02.577231 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3d8893a6-fea3-4fad-bb95-39018feb7b50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40426d2d806f4648a107abb2393b8d01'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:02.597531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1291/2556] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:58:02 2022] DELETE /identity/v3/users/40426d2d806f4648a107abb2393b8d01 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:58:02.651059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee3b31-1f55-4cb2-acb3-5f95d73e56a7 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:02.719352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfca56-27e0-40d1-aec6-1750c0fa7391 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:02.720057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfca56-27e0-40d1-aec6-1750c0fa7391 service swift] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:02.720730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfca56-27e0-40d1-aec6-1750c0fa7391 service swift] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:02.721066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfca56-27e0-40d1-aec6-1750c0fa7391 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:02.770918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:58:02.786029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfca56-27e0-40d1-aec6-1750c0fa7391 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:02.787648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfca56-27e0-40d1-aec6-1750c0fa7391 service swift] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:02.796328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1266/2557] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 16:58:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 16:58:02.879081 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8893a6-fea3-4fad-bb95-39018feb7b50 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:02.880984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3d8893a6-fea3-4fad-bb95-39018feb7b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:02.976550 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8893a6-fea3-4fad-bb95-39018feb7b50 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:02.983002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0226df7-1e7b-45a4-bcd5-72d0804378da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:02.984209 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0226df7-1e7b-45a4-bcd5-72d0804378da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:02.984753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0226df7-1e7b-45a4-bcd5-72d0804378da admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:02.985367 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0226df7-1e7b-45a4-bcd5-72d0804378da admin admin] PATH_INFO: `/v3/projects/032ab101adb94ba0921407b40b6beeea` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:02.987427 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0226df7-1e7b-45a4-bcd5-72d0804378da admin admin] RBAC: Authorizing `identity:delete_project(project_id=032ab101adb94ba0921407b40b6beeea)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:02.989877 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0226df7-1e7b-45a4-bcd5-72d0804378da admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:03.123858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0226df7-1e7b-45a4-bcd5-72d0804378da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '032ab101adb94ba0921407b40b6beeea', 'request_id': 'req-a0226df7-1e7b-45a4-bcd5-72d0804378da'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:03.138207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0226df7-1e7b-45a4-bcd5-72d0804378da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '032ab101adb94ba0921407b40b6beeea', 'request_id': 'req-a0226df7-1e7b-45a4-bcd5-72d0804378da'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:03.160204 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0226df7-1e7b-45a4-bcd5-72d0804378da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '032ab101adb94ba0921407b40b6beeea', 'request_id': 'req-a0226df7-1e7b-45a4-bcd5-72d0804378da'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:03.276911 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0226df7-1e7b-45a4-bcd5-72d0804378da admin admin] The token cache is being invalidate because project 032ab101adb94ba0921407b40b6beeea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:58:03.277582 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0226df7-1e7b-45a4-bcd5-72d0804378da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:03.277988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0226df7-1e7b-45a4-bcd5-72d0804378da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:58:03.279618 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1292/2558] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:58:02 2022] DELETE /identity/v3/projects/032ab101adb94ba0921407b40b6beeea => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:58:03.284974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbc58a-b11d-4a98-a646-e37253c8d478 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:03.285459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbc58a-b11d-4a98-a646-e37253c8d478 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:03.285559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbc58a-b11d-4a98-a646-e37253c8d478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:03.310270 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-17bbc58a-b11d-4a98-a646-e37253c8d478 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:58:03.312101 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-17bbc58a-b11d-4a98-a646-e37253c8d478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:03.380446 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1267/2559] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:58:03.387186 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dcbf3-f81e-4c95-8eaf-63182401d322 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:03.387547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dcbf3-f81e-4c95-8eaf-63182401d322 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:03.387944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dcbf3-f81e-4c95-8eaf-63182401d322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:03.411573 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:58:03.421895 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0e6dcbf3-f81e-4c95-8eaf-63182401d322 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:58:03.423970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0e6dcbf3-f81e-4c95-8eaf-63182401d322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:03.433552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1293/2560] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:58:03.437559 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbc58a-b11d-4a98-a646-e37253c8d478 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:03.449610 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbc58a-b11d-4a98-a646-e37253c8d478 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:03.457239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31021764-558b-4038-8045-e157a7008404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:03.457239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31021764-558b-4038-8045-e157a7008404 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:03.457239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31021764-558b-4038-8045-e157a7008404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:03.457239 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31021764-558b-4038-8045-e157a7008404 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:03.457644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31021764-558b-4038-8045-e157a7008404 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:58:03.457644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31021764-558b-4038-8045-e157a7008404 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:03.458600 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31021764-558b-4038-8045-e157a7008404 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:03.473830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1268/2561] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:58:03.478361 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dcbf3-f81e-4c95-8eaf-63182401d322 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:03.488159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dcbf3-f81e-4c95-8eaf-63182401d322 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:03.498461 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90e16c-5925-45d6-88af-09ad9d919f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:03.499348 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90e16c-5925-45d6-88af-09ad9d919f73 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:03.499638 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90e16c-5925-45d6-88af-09ad9d919f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:03.499918 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90e16c-5925-45d6-88af-09ad9d919f73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:03.500599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90e16c-5925-45d6-88af-09ad9d919f73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:03.507338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90e16c-5925-45d6-88af-09ad9d919f73 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:03.645635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1294/2562] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:58:03 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:58:03.649941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31021764-558b-4038-8045-e157a7008404 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:03.660947 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31021764-558b-4038-8045-e157a7008404 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:03.740424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e0a17-80f9-4ee3-8206-462526f5937a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:03.741122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e0a17-80f9-4ee3-8206-462526f5937a admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:03.741467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e0a17-80f9-4ee3-8206-462526f5937a admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:03.741805 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e0a17-80f9-4ee3-8206-462526f5937a admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:03.742502 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e0a17-80f9-4ee3-8206-462526f5937a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:03.745278 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e0a17-80f9-4ee3-8206-462526f5937a admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:03.881900 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1269/2563] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:58:03 2022] POST /identity/v3/users => generated 290 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:58:03.886089 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90e16c-5925-45d6-88af-09ad9d919f73 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:03.896792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90e16c-5925-45d6-88af-09ad9d919f73 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:03.902361 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1f3d5-a07f-4d36-ac38-2a475202b5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:03.903054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1f3d5-a07f-4d36-ac38-2a475202b5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:03.903417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1f3d5-a07f-4d36-ac38-2a475202b5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:03.903771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1f3d5-a07f-4d36-ac38-2a475202b5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:03.904515 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1f3d5-a07f-4d36-ac38-2a475202b5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:03.906187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1f3d5-a07f-4d36-ac38-2a475202b5f3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:03.921026 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1295/2564] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:58:03.922301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e0a17-80f9-4ee3-8206-462526f5937a admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:03.932794 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e0a17-80f9-4ee3-8206-462526f5937a admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:03.937685 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c9d4a-6ce9-47e3-ae5a-2c01ae2b9071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:03.938310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c9d4a-6ce9-47e3-ae5a-2c01ae2b9071 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:03.938596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c9d4a-6ce9-47e3-ae5a-2c01ae2b9071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:03.938875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c9d4a-6ce9-47e3-ae5a-2c01ae2b9071 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:03.939620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c9d4a-6ce9-47e3-ae5a-2c01ae2b9071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:03.941398 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c9d4a-6ce9-47e3-ae5a-2c01ae2b9071 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:03.953378 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1270/2565] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:58:03.958086 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1f3d5-a07f-4d36-ac38-2a475202b5f3 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:03.969098 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1f3d5-a07f-4d36-ac38-2a475202b5f3 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:03.974558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff1555-e862-47a4-98f6-160834aaf45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:03.975345 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff1555-e862-47a4-98f6-160834aaf45f admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:03.975652 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff1555-e862-47a4-98f6-160834aaf45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:03.976002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff1555-e862-47a4-98f6-160834aaf45f admin admin] PATH_INFO: `/v3/projects/805d6de24ff4415c89cba2d318ffd2f4/users/701be5a3cff34016ab22b6dc4d23d939/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:03.994672 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff1555-e862-47a4-98f6-160834aaf45f admin admin] RBAC: Authorizing `identity:create_grant(project_id=805d6de24ff4415c89cba2d318ffd2f4, user_id=701be5a3cff34016ab22b6dc4d23d939, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:03.997392 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff1555-e862-47a4-98f6-160834aaf45f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:04.117608 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1296/2566] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:58:03 2022] PUT /identity/v3/projects/805d6de24ff4415c89cba2d318ffd2f4/users/701be5a3cff34016ab22b6dc4d23d939/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:58:04.123170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abaa62-1d7b-42a7-89ef-129d2efde54c None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:04.123648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abaa62-1d7b-42a7-89ef-129d2efde54c None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:04.123932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abaa62-1d7b-42a7-89ef-129d2efde54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:04.144609 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-a7abaa62-1d7b-42a7-89ef-129d2efde54c None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:58:04.145809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a7abaa62-1d7b-42a7-89ef-129d2efde54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:04.189760 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:58:04.207995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1271/2567] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:58:04.218735 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff1555-e862-47a4-98f6-160834aaf45f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:04.221019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-eaff1555-e862-47a4-98f6-160834aaf45f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:04.320855 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff1555-e862-47a4-98f6-160834aaf45f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:04.326895 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d625b-d6db-46fc-a278-32e1561ada05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:04.327824 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d625b-d6db-46fc-a278-32e1561ada05 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:04.328270 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d625b-d6db-46fc-a278-32e1561ada05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:04.328769 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d625b-d6db-46fc-a278-32e1561ada05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:04.342452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d625b-d6db-46fc-a278-32e1561ada05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:04.344146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d625b-d6db-46fc-a278-32e1561ada05 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:04.356912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1297/2568] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:58:08.542221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abaa62-1d7b-42a7-89ef-129d2efde54c None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:08.553463 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abaa62-1d7b-42a7-89ef-129d2efde54c None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:08.561648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c550c7-c793-4640-95d9-eaa6d6827e02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:08.561648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c550c7-c793-4640-95d9-eaa6d6827e02 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:08.561648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c550c7-c793-4640-95d9-eaa6d6827e02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:08.561648 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c550c7-c793-4640-95d9-eaa6d6827e02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:08.563772 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b4c550c7-c793-4640-95d9-eaa6d6827e02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:08.680668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c550c7-c793-4640-95d9-eaa6d6827e02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:08.682122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c550c7-c793-4640-95d9-eaa6d6827e02 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:08.708485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1272/2569] 10.222.0.70 () {60 vars in 1396 bytes} [Wed Aug 24 16:58:08 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:58:11.931280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c4bbd-ccda-4e5c-8bea-74ffe3891781 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:11.931889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c4bbd-ccda-4e5c-8bea-74ffe3891781 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:11.931889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c4bbd-ccda-4e5c-8bea-74ffe3891781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:11.963842 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-360c4bbd-ccda-4e5c-8bea-74ffe3891781 None None] MFA Rules not processed for user `701be5a3cff34016ab22b6dc4d23d939`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:58:11.965122 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-360c4bbd-ccda-4e5c-8bea-74ffe3891781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:12.034816 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1298/2570] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:58:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:58:12.039387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c550c7-c793-4640-95d9-eaa6d6827e02 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:12.051222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c550c7-c793-4640-95d9-eaa6d6827e02 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:12.056607 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee70e2-9cfc-4d68-b0d6-f5d6d5636bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:12.057233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee70e2-9cfc-4d68-b0d6-f5d6d5636bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:12.057686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee70e2-9cfc-4d68-b0d6-f5d6d5636bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:12.057797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee70e2-9cfc-4d68-b0d6-f5d6d5636bf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:12.058391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee70e2-9cfc-4d68-b0d6-f5d6d5636bf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:12.060316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee70e2-9cfc-4d68-b0d6-f5d6d5636bf7 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:12.170681 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1273/2571] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:58:12 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:58:12.174741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c4bbd-ccda-4e5c-8bea-74ffe3891781 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:12.186561 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c4bbd-ccda-4e5c-8bea-74ffe3891781 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:12.219592 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:58:12.243390 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf970bd2-b18c-4d06-99be-9436dd13715b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:12.244001 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf970bd2-b18c-4d06-99be-9436dd13715b admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:12.244281 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf970bd2-b18c-4d06-99be-9436dd13715b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:12.244596 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf970bd2-b18c-4d06-99be-9436dd13715b admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:12.245230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf970bd2-b18c-4d06-99be-9436dd13715b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:12.247252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf970bd2-b18c-4d06-99be-9436dd13715b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:12.344537 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1299/2572] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:58:12 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:58:12.348490 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee70e2-9cfc-4d68-b0d6-f5d6d5636bf7 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:12.359562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee70e2-9cfc-4d68-b0d6-f5d6d5636bf7 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:12.364134 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67076862-97c8-4de1-8a49-694f4b42ad4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:12.364755 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67076862-97c8-4de1-8a49-694f4b42ad4f admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:12.365041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67076862-97c8-4de1-8a49-694f4b42ad4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:12.365312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67076862-97c8-4de1-8a49-694f4b42ad4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:12.366029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67076862-97c8-4de1-8a49-694f4b42ad4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:12.367623 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67076862-97c8-4de1-8a49-694f4b42ad4f admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:12.378278 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1274/2573] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:58:12.384547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf970bd2-b18c-4d06-99be-9436dd13715b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:12.395182 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf970bd2-b18c-4d06-99be-9436dd13715b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:12.400633 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890c8fa-1387-4cec-9bfd-262a2c114611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:12.401306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890c8fa-1387-4cec-9bfd-262a2c114611 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:12.401586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890c8fa-1387-4cec-9bfd-262a2c114611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:12.401866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890c8fa-1387-4cec-9bfd-262a2c114611 admin admin] PATH_INFO: `/v3/projects/a517d3953b9f4ef4ae73e2fc3d83ec4f/users/cb6f2c69360d4b8a9b242415035b335c/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:12.418505 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890c8fa-1387-4cec-9bfd-262a2c114611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a517d3953b9f4ef4ae73e2fc3d83ec4f, user_id=cb6f2c69360d4b8a9b242415035b335c, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:12.421959 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890c8fa-1387-4cec-9bfd-262a2c114611 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:12.516126 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1300/2574] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:58:12 2022] PUT /identity/v3/projects/a517d3953b9f4ef4ae73e2fc3d83ec4f/users/cb6f2c69360d4b8a9b242415035b335c/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:58:12.520054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67076862-97c8-4de1-8a49-694f4b42ad4f admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:12.530561 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67076862-97c8-4de1-8a49-694f4b42ad4f admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:12.600666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d49a22-0e13-4f14-a95b-c686519b55ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:12.601243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d49a22-0e13-4f14-a95b-c686519b55ce admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:12.601528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d49a22-0e13-4f14-a95b-c686519b55ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:12.601808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d49a22-0e13-4f14-a95b-c686519b55ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:12.602490 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d49a22-0e13-4f14-a95b-c686519b55ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:12.604164 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d49a22-0e13-4f14-a95b-c686519b55ce admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:12.615308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1275/2575] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:58:12.619445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890c8fa-1387-4cec-9bfd-262a2c114611 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:12.630388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890c8fa-1387-4cec-9bfd-262a2c114611 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:12.635775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6126f58-9942-477b-b2cb-a2f5d3c96dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:12.636476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6126f58-9942-477b-b2cb-a2f5d3c96dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:12.636763 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6126f58-9942-477b-b2cb-a2f5d3c96dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:12.637041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6126f58-9942-477b-b2cb-a2f5d3c96dc1 admin admin] PATH_INFO: `/v3/projects/a517d3953b9f4ef4ae73e2fc3d83ec4f/users/cb6f2c69360d4b8a9b242415035b335c/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:12.638961 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6126f58-9942-477b-b2cb-a2f5d3c96dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a517d3953b9f4ef4ae73e2fc3d83ec4f, user_id=cb6f2c69360d4b8a9b242415035b335c, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:12.641698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6126f58-9942-477b-b2cb-a2f5d3c96dc1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:12.652793 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1301/2576] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:58:12 2022] PUT /identity/v3/projects/a517d3953b9f4ef4ae73e2fc3d83ec4f/users/cb6f2c69360d4b8a9b242415035b335c/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:58:18.910124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8395155-282d-4743-abbc-0426574d2b1b None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:18.910802 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8395155-282d-4743-abbc-0426574d2b1b None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:18.911181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8395155-282d-4743-abbc-0426574d2b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:18.953508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-d8395155-282d-4743-abbc-0426574d2b1b None None] MFA Rules not processed for user `cb6f2c69360d4b8a9b242415035b335c`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:58:18.954832 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d8395155-282d-4743-abbc-0426574d2b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:18.988848 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:58:19.049039 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1276/2577] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:58:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:58:19.062842 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6126f58-9942-477b-b2cb-a2f5d3c96dc1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:19.064889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c6126f58-9942-477b-b2cb-a2f5d3c96dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:19.147124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6126f58-9942-477b-b2cb-a2f5d3c96dc1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:19.153446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4f087-8813-4ee0-a97d-67baf52cc6b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:19.154174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4f087-8813-4ee0-a97d-67baf52cc6b1 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:19.154513 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4f087-8813-4ee0-a97d-67baf52cc6b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:19.154794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4f087-8813-4ee0-a97d-67baf52cc6b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:19.214315 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4f087-8813-4ee0-a97d-67baf52cc6b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:19.216047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4f087-8813-4ee0-a97d-67baf52cc6b1 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:19.225854 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1302/2578] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:58:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:58:23.782135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b55637-80d7-4f21-bd16-dccff5c82b85 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:23.782611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b55637-80d7-4f21-bd16-dccff5c82b85 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:23.782716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b55637-80d7-4f21-bd16-dccff5c82b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:23.835805 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-57b55637-80d7-4f21-bd16-dccff5c82b85 None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:58:23.837060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-57b55637-80d7-4f21-bd16-dccff5c82b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:23.919478 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1277/2579] 10.222.0.70 () {58 vars in 967 bytes} [Wed Aug 24 16:58:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:58:23.933391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4f087-8813-4ee0-a97d-67baf52cc6b1 service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:23.935329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f1c4f087-8813-4ee0-a97d-67baf52cc6b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:23.986347 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:58:24.063162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4f087-8813-4ee0-a97d-67baf52cc6b1 service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:24.068242 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ceafe-3282-4c2d-811e-dcc6cb40b112 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:24.068881 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ceafe-3282-4c2d-811e-dcc6cb40b112 service swift] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:24.069278 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ceafe-3282-4c2d-811e-dcc6cb40b112 service swift] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:24.069564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ceafe-3282-4c2d-811e-dcc6cb40b112 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:24.078285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ceafe-3282-4c2d-811e-dcc6cb40b112 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:24.079840 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ceafe-3282-4c2d-811e-dcc6cb40b112 service swift] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:24.088173 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1303/2580] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 16:58:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:58:33.917069 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=61 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:58:34.084796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=30 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:58:56.649775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da2335-09ab-4fb2-b58d-227a2dcd6b68 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:56.653416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da2335-09ab-4fb2-b58d-227a2dcd6b68 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:56.653416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da2335-09ab-4fb2-b58d-227a2dcd6b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:56.669582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-09da2335-09ab-4fb2-b58d-227a2dcd6b68 None None] MFA Rules not processed for user `701be5a3cff34016ab22b6dc4d23d939`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:58:56.671380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-09da2335-09ab-4fb2-b58d-227a2dcd6b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:56.678336 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1278/2581] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:58:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:58:56.991398 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ceafe-3282-4c2d-811e-dcc6cb40b112 service swift] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:57.046903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ceafe-3282-4c2d-811e-dcc6cb40b112 service swift] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:57.050534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8aae5-f2a3-46ff-a0a5-7a2ddd18b68e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:57.051169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8aae5-f2a3-46ff-a0a5-7a2ddd18b68e service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:57.051451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8aae5-f2a3-46ff-a0a5-7a2ddd18b68e service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:57.051726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8aae5-f2a3-46ff-a0a5-7a2ddd18b68e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:57.053526 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fed8aae5-f2a3-46ff-a0a5-7a2ddd18b68e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:57.062969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8aae5-f2a3-46ff-a0a5-7a2ddd18b68e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:57.064583 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8aae5-f2a3-46ff-a0a5-7a2ddd18b68e service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:57.072709 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1304/2582] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:58:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:58:57.844633 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da2335-09ab-4fb2-b58d-227a2dcd6b68 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:58:57.846033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-09da2335-09ab-4fb2-b58d-227a2dcd6b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:57.921018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da2335-09ab-4fb2-b58d-227a2dcd6b68 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:58:57.925993 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6ae0e-16c8-4bed-b99a-7ef4c3afc064 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:58:57.926596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6ae0e-16c8-4bed-b99a-7ef4c3afc064 service placement] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:58:57.926876 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6ae0e-16c8-4bed-b99a-7ef4c3afc064 service placement] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:58:57.927146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6ae0e-16c8-4bed-b99a-7ef4c3afc064 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:58:57.928897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-87b6ae0e-16c8-4bed-b99a-7ef4c3afc064 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:58:57.940824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6ae0e-16c8-4bed-b99a-7ef4c3afc064 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:58:57.942387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6ae0e-16c8-4bed-b99a-7ef4c3afc064 service placement] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:58:57.952819 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1279/2583] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:58:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:59:01.236062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8aae5-f2a3-46ff-a0a5-7a2ddd18b68e service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:01.245034 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fed8aae5-f2a3-46ff-a0a5-7a2ddd18b68e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:01.259441 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8aae5-f2a3-46ff-a0a5-7a2ddd18b68e service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:01.263516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccd474-ed1c-41c9-aa6d-70925860b119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:01.264105 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccd474-ed1c-41c9-aa6d-70925860b119 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:01.264477 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccd474-ed1c-41c9-aa6d-70925860b119 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:01.264845 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccd474-ed1c-41c9-aa6d-70925860b119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:01.266691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-12ccd474-ed1c-41c9-aa6d-70925860b119 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:01.280862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccd474-ed1c-41c9-aa6d-70925860b119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:01.282573 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccd474-ed1c-41c9-aa6d-70925860b119 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:01.292908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1305/2584] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:59:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:59:07.947977 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=35 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:59:11.288770 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=32 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:59:28.697910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6ae0e-16c8-4bed-b99a-7ef4c3afc064 service placement] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:28.707916 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6ae0e-16c8-4bed-b99a-7ef4c3afc064 service placement] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:28.711639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1dea7-3414-4f62-894f-9d769746d8dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:28.712283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1dea7-3414-4f62-894f-9d769746d8dc service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:28.712592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1dea7-3414-4f62-894f-9d769746d8dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:28.712895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1dea7-3414-4f62-894f-9d769746d8dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:28.722617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1dea7-3414-4f62-894f-9d769746d8dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:28.724038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1dea7-3414-4f62-894f-9d769746d8dc service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:28.734090 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1280/2585] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:59:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:59:36.597021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccd474-ed1c-41c9-aa6d-70925860b119 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:36.605539 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccd474-ed1c-41c9-aa6d-70925860b119 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:36.609671 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd61f56-6c3f-42fc-aca5-9ce86bc9ba10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:36.610261 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd61f56-6c3f-42fc-aca5-9ce86bc9ba10 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:36.610541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd61f56-6c3f-42fc-aca5-9ce86bc9ba10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:36.610815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd61f56-6c3f-42fc-aca5-9ce86bc9ba10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:36.612643 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dcd61f56-6c3f-42fc-aca5-9ce86bc9ba10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:36.678979 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd61f56-6c3f-42fc-aca5-9ce86bc9ba10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:36.680432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd61f56-6c3f-42fc-aca5-9ce86bc9ba10 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:36.690137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1306/2586] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:59:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:59:38.729489 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=6 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:59:40.320360 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1dea7-3414-4f62-894f-9d769746d8dc service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:40.322072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-16d1dea7-3414-4f62-894f-9d769746d8dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:40.337309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1dea7-3414-4f62-894f-9d769746d8dc service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:40.342632 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c15cc-7787-4c63-abfb-d55a9029c0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:40.343466 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c15cc-7787-4c63-abfb-d55a9029c0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:40.343752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c15cc-7787-4c63-abfb-d55a9029c0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:40.344033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c15cc-7787-4c63-abfb-d55a9029c0e7 admin admin] PATH_INFO: `/v3/users/88efd5eb09064aa2aa89e6d9580c6476` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:40.358670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c15cc-7787-4c63-abfb-d55a9029c0e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88efd5eb09064aa2aa89e6d9580c6476)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:40.362146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c15cc-7787-4c63-abfb-d55a9029c0e7 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:40.522910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-8c1c15cc-7787-4c63-abfb-d55a9029c0e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88efd5eb09064aa2aa89e6d9580c6476'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:40.523570 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-8c1c15cc-7787-4c63-abfb-d55a9029c0e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88efd5eb09064aa2aa89e6d9580c6476'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:40.524259 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-8c1c15cc-7787-4c63-abfb-d55a9029c0e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88efd5eb09064aa2aa89e6d9580c6476'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:40.531407 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-8c1c15cc-7787-4c63-abfb-d55a9029c0e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88efd5eb09064aa2aa89e6d9580c6476'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:40.571748 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-8c1c15cc-7787-4c63-abfb-d55a9029c0e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88efd5eb09064aa2aa89e6d9580c6476'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:40.589272 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1281/2587] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:59:40 2022] DELETE /identity/v3/users/88efd5eb09064aa2aa89e6d9580c6476 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:59:40.830772 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd61f56-6c3f-42fc-aca5-9ce86bc9ba10 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:40.844109 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd61f56-6c3f-42fc-aca5-9ce86bc9ba10 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:40.904897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217daed-c574-4966-88b5-9641129e3abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:40.905540 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217daed-c574-4966-88b5-9641129e3abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:40.906328 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217daed-c574-4966-88b5-9641129e3abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:40.906623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217daed-c574-4966-88b5-9641129e3abc admin admin] PATH_INFO: `/v3/projects/6b5b284e0b47422592057492601db382` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:40.917376 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217daed-c574-4966-88b5-9641129e3abc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b5b284e0b47422592057492601db382)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:40.919330 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217daed-c574-4966-88b5-9641129e3abc admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:41.027954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5217daed-c574-4966-88b5-9641129e3abc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b5b284e0b47422592057492601db382', 'request_id': 'req-5217daed-c574-4966-88b5-9641129e3abc'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:41.035314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5217daed-c574-4966-88b5-9641129e3abc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b5b284e0b47422592057492601db382', 'request_id': 'req-5217daed-c574-4966-88b5-9641129e3abc'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:41.040930 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5217daed-c574-4966-88b5-9641129e3abc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b5b284e0b47422592057492601db382', 'request_id': 'req-5217daed-c574-4966-88b5-9641129e3abc'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:41.142759 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5217daed-c574-4966-88b5-9641129e3abc admin admin] The token cache is being invalidate because project 6b5b284e0b47422592057492601db382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:59:41.143047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5217daed-c574-4966-88b5-9641129e3abc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:41.143661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5217daed-c574-4966-88b5-9641129e3abc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:41.145183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1307/2588] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:59:40 2022] DELETE /identity/v3/projects/6b5b284e0b47422592057492601db382 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:59:41.150073 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0b76c-1f1c-4ca0-8778-cec920526509 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:41.150360 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0b76c-1f1c-4ca0-8778-cec920526509 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:41.150638 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0b76c-1f1c-4ca0-8778-cec920526509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:41.169717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-68a0b76c-1f1c-4ca0-8778-cec920526509 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:59:41.171388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-68a0b76c-1f1c-4ca0-8778-cec920526509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:41.230241 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1282/2589] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:59:41.238434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40fb2c-80bf-40fb-9b43-f6a8a86173ab None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:41.238728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40fb2c-80bf-40fb-9b43-f6a8a86173ab None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:41.239007 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40fb2c-80bf-40fb-9b43-f6a8a86173ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:41.258951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-eb40fb2c-80bf-40fb-9b43-f6a8a86173ab None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:59:41.260174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-eb40fb2c-80bf-40fb-9b43-f6a8a86173ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:41.268427 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1308/2590] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:59:41.272576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0b76c-1f1c-4ca0-8778-cec920526509 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:41.282824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0b76c-1f1c-4ca0-8778-cec920526509 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:41.287655 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba487a4-5db0-4187-a8c7-09764dba8a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:41.288297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba487a4-5db0-4187-a8c7-09764dba8a8a admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:41.290254 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba487a4-5db0-4187-a8c7-09764dba8a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:41.290254 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba487a4-5db0-4187-a8c7-09764dba8a8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:41.290254 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba487a4-5db0-4187-a8c7-09764dba8a8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:59:41.290254 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba487a4-5db0-4187-a8c7-09764dba8a8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:41.291397 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba487a4-5db0-4187-a8c7-09764dba8a8a admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:41.303752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1283/2591] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:59:41.308142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40fb2c-80bf-40fb-9b43-f6a8a86173ab None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:41.317953 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40fb2c-80bf-40fb-9b43-f6a8a86173ab None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:41.322690 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d1acc-9913-438a-86c1-5b8dcb95e5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:41.323296 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d1acc-9913-438a-86c1-5b8dcb95e5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:41.323577 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d1acc-9913-438a-86c1-5b8dcb95e5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:41.323850 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d1acc-9913-438a-86c1-5b8dcb95e5d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:41.324543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d1acc-9913-438a-86c1-5b8dcb95e5d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:41.326488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d1acc-9913-438a-86c1-5b8dcb95e5d1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:41.332001 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:59:41.434338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1309/2592] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:59:41 2022] POST /identity/v3/projects => generated 403 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:59:41.435464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba487a4-5db0-4187-a8c7-09764dba8a8a admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:41.445770 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba487a4-5db0-4187-a8c7-09764dba8a8a admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:41.492290 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:59:41.508754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a18a4-b137-4ce8-bb2d-02975de215ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:41.509468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a18a4-b137-4ce8-bb2d-02975de215ab admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:41.509979 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a18a4-b137-4ce8-bb2d-02975de215ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:41.510382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a18a4-b137-4ce8-bb2d-02975de215ab admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:41.511080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a18a4-b137-4ce8-bb2d-02975de215ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:41.513221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a18a4-b137-4ce8-bb2d-02975de215ab admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:41.624481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1284/2593] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:59:41 2022] POST /identity/v3/users => generated 302 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:59:41.630781 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d1acc-9913-438a-86c1-5b8dcb95e5d1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:41.640326 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d1acc-9913-438a-86c1-5b8dcb95e5d1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:41.646183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8128f-5e7e-4a97-b7f9-e986faf0d08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:41.647089 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8128f-5e7e-4a97-b7f9-e986faf0d08d admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:41.647475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8128f-5e7e-4a97-b7f9-e986faf0d08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:41.647828 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8128f-5e7e-4a97-b7f9-e986faf0d08d admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:41.648596 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e8128f-5e7e-4a97-b7f9-e986faf0d08d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:41.650540 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e8128f-5e7e-4a97-b7f9-e986faf0d08d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:41.671736 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1310/2594] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:59:41.673014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a18a4-b137-4ce8-bb2d-02975de215ab admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:41.683638 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a18a4-b137-4ce8-bb2d-02975de215ab admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:41.688684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7debc-1ad6-4b6d-a180-8d0fab6fb052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:41.689289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7debc-1ad6-4b6d-a180-8d0fab6fb052 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:41.689568 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7debc-1ad6-4b6d-a180-8d0fab6fb052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:41.689840 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7debc-1ad6-4b6d-a180-8d0fab6fb052 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:41.690489 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7debc-1ad6-4b6d-a180-8d0fab6fb052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:41.692148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7debc-1ad6-4b6d-a180-8d0fab6fb052 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:41.706921 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1285/2595] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:59:41.708043 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8128f-5e7e-4a97-b7f9-e986faf0d08d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:41.718887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8128f-5e7e-4a97-b7f9-e986faf0d08d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:41.724885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12541b9c-4eea-40d7-b075-11101f083f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:41.730682 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12541b9c-4eea-40d7-b075-11101f083f7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:41.731040 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12541b9c-4eea-40d7-b075-11101f083f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:41.733007 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12541b9c-4eea-40d7-b075-11101f083f7f admin admin] PATH_INFO: `/v3/projects/bbfa9556e9b34c34a73e113005011d5b/users/814afc78a1ac4e648161c65612a17f20/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:41.747931 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12541b9c-4eea-40d7-b075-11101f083f7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbfa9556e9b34c34a73e113005011d5b, user_id=814afc78a1ac4e648161c65612a17f20, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:41.750733 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12541b9c-4eea-40d7-b075-11101f083f7f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:41.830265 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1311/2596] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:59:41 2022] PUT /identity/v3/projects/bbfa9556e9b34c34a73e113005011d5b/users/814afc78a1ac4e648161c65612a17f20/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:59:41.835775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad5420-6e59-4514-96c6-20cade09d181 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:41.836068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad5420-6e59-4514-96c6-20cade09d181 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:41.836338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad5420-6e59-4514-96c6-20cade09d181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:41.856431 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-73ad5420-6e59-4514-96c6-20cade09d181 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:59:41.857862 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-73ad5420-6e59-4514-96c6-20cade09d181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:41.918315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1286/2597] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:59:41.928121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12541b9c-4eea-40d7-b075-11101f083f7f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:41.929957 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-12541b9c-4eea-40d7-b075-11101f083f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:41.987624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12541b9c-4eea-40d7-b075-11101f083f7f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:41.991190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451877c-cdcf-4120-b2db-388e31246fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:41.991797 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451877c-cdcf-4120-b2db-388e31246fdf service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:41.992118 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451877c-cdcf-4120-b2db-388e31246fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:41.992394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451877c-cdcf-4120-b2db-388e31246fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:42.002688 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451877c-cdcf-4120-b2db-388e31246fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:42.004143 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451877c-cdcf-4120-b2db-388e31246fdf service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:42.014448 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1312/2598] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:59:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:59:43.173492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b7c6d-1c6c-40ff-b263-71dc12ceaa84 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:43.174029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b7c6d-1c6c-40ff-b263-71dc12ceaa84 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:43.174029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b7c6d-1c6c-40ff-b263-71dc12ceaa84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:43.202569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-8f9b7c6d-1c6c-40ff-b263-71dc12ceaa84 None None] MFA Rules not processed for user `814afc78a1ac4e648161c65612a17f20`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:59:43.203797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-8f9b7c6d-1c6c-40ff-b263-71dc12ceaa84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:43.273330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1287/2599] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:59:43 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:59:43.277821 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451877c-cdcf-4120-b2db-388e31246fdf service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:43.288311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451877c-cdcf-4120-b2db-388e31246fdf service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:43.293016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857256a4-9422-4625-9ae0-54a881a85a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:43.293614 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857256a4-9422-4625-9ae0-54a881a85a5f admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:43.293899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857256a4-9422-4625-9ae0-54a881a85a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:43.294197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857256a4-9422-4625-9ae0-54a881a85a5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:43.295318 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857256a4-9422-4625-9ae0-54a881a85a5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:43.297291 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857256a4-9422-4625-9ae0-54a881a85a5f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:43.367066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1313/2600] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:59:43 2022] POST /identity/v3/projects => generated 403 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:59:43.371056 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b7c6d-1c6c-40ff-b263-71dc12ceaa84 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:43.382592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b7c6d-1c6c-40ff-b263-71dc12ceaa84 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:43.432026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:59:43.438303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3478bb6-8383-487b-ac4a-c2870e60a048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:43.442137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3478bb6-8383-487b-ac4a-c2870e60a048 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:43.442137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3478bb6-8383-487b-ac4a-c2870e60a048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:43.442137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3478bb6-8383-487b-ac4a-c2870e60a048 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:43.442137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3478bb6-8383-487b-ac4a-c2870e60a048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:43.442137 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3478bb6-8383-487b-ac4a-c2870e60a048 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:43.514740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1288/2601] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:59:43 2022] POST /identity/v3/users => generated 308 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:59:43.518504 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857256a4-9422-4625-9ae0-54a881a85a5f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:43.528712 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857256a4-9422-4625-9ae0-54a881a85a5f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:43.533189 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b6add-9c3f-46c6-899b-c42294eec4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:43.533810 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b6add-9c3f-46c6-899b-c42294eec4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:43.534070 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b6add-9c3f-46c6-899b-c42294eec4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:43.534345 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b6add-9c3f-46c6-899b-c42294eec4a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:43.534984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b6add-9c3f-46c6-899b-c42294eec4a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:43.536619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b6add-9c3f-46c6-899b-c42294eec4a8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:43.548543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1314/2602] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:59:43.552821 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3478bb6-8383-487b-ac4a-c2870e60a048 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:43.566166 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3478bb6-8383-487b-ac4a-c2870e60a048 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:43.570742 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5152f07-9300-4cd1-806d-a8faf9019207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:43.571468 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5152f07-9300-4cd1-806d-a8faf9019207 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:43.571780 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5152f07-9300-4cd1-806d-a8faf9019207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:43.572102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5152f07-9300-4cd1-806d-a8faf9019207 admin admin] PATH_INFO: `/v3/projects/c66f88d2acc24b2198099c12f463e0c9/users/2f1c72bb0e2f481fbd5abfdc4d1afd4c/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:43.588742 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5152f07-9300-4cd1-806d-a8faf9019207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c66f88d2acc24b2198099c12f463e0c9, user_id=2f1c72bb0e2f481fbd5abfdc4d1afd4c, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:43.591413 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5152f07-9300-4cd1-806d-a8faf9019207 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:43.691022 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1289/2603] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:59:43 2022] PUT /identity/v3/projects/c66f88d2acc24b2198099c12f463e0c9/users/2f1c72bb0e2f481fbd5abfdc4d1afd4c/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:59:43.694992 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b6add-9c3f-46c6-899b-c42294eec4a8 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:43.705274 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b6add-9c3f-46c6-899b-c42294eec4a8 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:43.759645 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e6152-9fde-409c-a35d-bb2b87022be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:43.760286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e6152-9fde-409c-a35d-bb2b87022be5 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:43.760602 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e6152-9fde-409c-a35d-bb2b87022be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:43.760881 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e6152-9fde-409c-a35d-bb2b87022be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:43.761526 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e6152-9fde-409c-a35d-bb2b87022be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:43.763115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e6152-9fde-409c-a35d-bb2b87022be5 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:43.773287 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1315/2604] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:59:43.777309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5152f07-9300-4cd1-806d-a8faf9019207 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:43.787411 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5152f07-9300-4cd1-806d-a8faf9019207 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:43.791941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0cfe5-3865-4549-8180-c54954c354cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:43.792622 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0cfe5-3865-4549-8180-c54954c354cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:43.792925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0cfe5-3865-4549-8180-c54954c354cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:43.793179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0cfe5-3865-4549-8180-c54954c354cc admin admin] PATH_INFO: `/v3/projects/c66f88d2acc24b2198099c12f463e0c9/users/2f1c72bb0e2f481fbd5abfdc4d1afd4c/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:43.795119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0cfe5-3865-4549-8180-c54954c354cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c66f88d2acc24b2198099c12f463e0c9, user_id=2f1c72bb0e2f481fbd5abfdc4d1afd4c, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:43.797861 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0cfe5-3865-4549-8180-c54954c354cc admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:43.808285 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1290/2605] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:59:43 2022] PUT /identity/v3/projects/c66f88d2acc24b2198099c12f463e0c9/users/2f1c72bb0e2f481fbd5abfdc4d1afd4c/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:59:45.030658 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901dcfee-94a2-4931-afa6-d5014c33dd69 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:45.033576 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901dcfee-94a2-4931-afa6-d5014c33dd69 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:45.033576 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901dcfee-94a2-4931-afa6-d5014c33dd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:45.059178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-901dcfee-94a2-4931-afa6-d5014c33dd69 None None] MFA Rules not processed for user `2f1c72bb0e2f481fbd5abfdc4d1afd4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:59:45.061161 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-901dcfee-94a2-4931-afa6-d5014c33dd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:45.066744 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:59:45.130505 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1316/2606] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:59:45 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:59:46.453102 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0cfe5-3865-4549-8180-c54954c354cc admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:46.520763 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0cfe5-3865-4549-8180-c54954c354cc admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:46.525258 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb2513-d81f-413a-a06b-c5dfa4c2994c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:46.525883 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb2513-d81f-413a-a06b-c5dfa4c2994c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:46.526169 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb2513-d81f-413a-a06b-c5dfa4c2994c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:46.526443 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb2513-d81f-413a-a06b-c5dfa4c2994c admin admin] PATH_INFO: `/v3/users/814afc78a1ac4e648161c65612a17f20` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:46.527552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb2513-d81f-413a-a06b-c5dfa4c2994c admin admin] RBAC: Authorizing `identity:delete_user(user_id=814afc78a1ac4e648161c65612a17f20)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:46.530066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb2513-d81f-413a-a06b-c5dfa4c2994c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:46.744046 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-18cb2513-d81f-413a-a06b-c5dfa4c2994c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '814afc78a1ac4e648161c65612a17f20'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:46.744717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-18cb2513-d81f-413a-a06b-c5dfa4c2994c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '814afc78a1ac4e648161c65612a17f20'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:46.745320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-18cb2513-d81f-413a-a06b-c5dfa4c2994c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '814afc78a1ac4e648161c65612a17f20'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:46.752332 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-18cb2513-d81f-413a-a06b-c5dfa4c2994c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '814afc78a1ac4e648161c65612a17f20'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:46.828707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-18cb2513-d81f-413a-a06b-c5dfa4c2994c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '814afc78a1ac4e648161c65612a17f20'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:46.845990 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1291/2607] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:59:46 2022] DELETE /identity/v3/users/814afc78a1ac4e648161c65612a17f20 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:59:47.070488 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901dcfee-94a2-4931-afa6-d5014c33dd69 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:47.081128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901dcfee-94a2-4931-afa6-d5014c33dd69 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:47.136993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883364d-c9db-4298-ac44-5f351f93551d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:47.137619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883364d-c9db-4298-ac44-5f351f93551d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:47.137896 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883364d-c9db-4298-ac44-5f351f93551d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:47.138169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883364d-c9db-4298-ac44-5f351f93551d admin admin] PATH_INFO: `/v3/projects/bbfa9556e9b34c34a73e113005011d5b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:47.139289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883364d-c9db-4298-ac44-5f351f93551d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbfa9556e9b34c34a73e113005011d5b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:47.141221 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883364d-c9db-4298-ac44-5f351f93551d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:47.248258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9883364d-c9db-4298-ac44-5f351f93551d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbfa9556e9b34c34a73e113005011d5b', 'request_id': 'req-9883364d-c9db-4298-ac44-5f351f93551d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:47.254902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9883364d-c9db-4298-ac44-5f351f93551d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbfa9556e9b34c34a73e113005011d5b', 'request_id': 'req-9883364d-c9db-4298-ac44-5f351f93551d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:47.261667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9883364d-c9db-4298-ac44-5f351f93551d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbfa9556e9b34c34a73e113005011d5b', 'request_id': 'req-9883364d-c9db-4298-ac44-5f351f93551d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:47.348627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9883364d-c9db-4298-ac44-5f351f93551d admin admin] The token cache is being invalidate because project bbfa9556e9b34c34a73e113005011d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:59:47.348848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9883364d-c9db-4298-ac44-5f351f93551d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:47.349461 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9883364d-c9db-4298-ac44-5f351f93551d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:47.350940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1317/2608] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:59:47 2022] DELETE /identity/v3/projects/bbfa9556e9b34c34a73e113005011d5b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:59:47.354984 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb2513-d81f-413a-a06b-c5dfa4c2994c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:47.356473 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-18cb2513-d81f-413a-a06b-c5dfa4c2994c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:47.369881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb2513-d81f-413a-a06b-c5dfa4c2994c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:47.396520 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:59:47.426898 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a973833-22d4-441b-a918-35cd59794a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:47.427521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a973833-22d4-441b-a918-35cd59794a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:47.427804 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a973833-22d4-441b-a918-35cd59794a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:47.428080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a973833-22d4-441b-a918-35cd59794a89 admin admin] PATH_INFO: `/v3/users/2f1c72bb0e2f481fbd5abfdc4d1afd4c` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:47.429245 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a973833-22d4-441b-a918-35cd59794a89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f1c72bb0e2f481fbd5abfdc4d1afd4c)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:47.431175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a973833-22d4-441b-a918-35cd59794a89 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:47.618828 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6a973833-22d4-441b-a918-35cd59794a89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f1c72bb0e2f481fbd5abfdc4d1afd4c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:47.623771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6a973833-22d4-441b-a918-35cd59794a89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f1c72bb0e2f481fbd5abfdc4d1afd4c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:47.623771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6a973833-22d4-441b-a918-35cd59794a89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f1c72bb0e2f481fbd5abfdc4d1afd4c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:47.627143 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6a973833-22d4-441b-a918-35cd59794a89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f1c72bb0e2f481fbd5abfdc4d1afd4c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:47.690505 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-6a973833-22d4-441b-a918-35cd59794a89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f1c72bb0e2f481fbd5abfdc4d1afd4c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:47.708389 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1292/2609] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 16:59:47 2022] DELETE /identity/v3/users/2f1c72bb0e2f481fbd5abfdc4d1afd4c => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:59:47.952738 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883364d-c9db-4298-ac44-5f351f93551d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:47.953501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9883364d-c9db-4298-ac44-5f351f93551d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:47.975824 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883364d-c9db-4298-ac44-5f351f93551d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:48.008039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:59:48.033832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a520e-813b-461a-a1a9-804320d4ce68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:48.034679 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a520e-813b-461a-a1a9-804320d4ce68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:48.035020 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a520e-813b-461a-a1a9-804320d4ce68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:48.035487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a520e-813b-461a-a1a9-804320d4ce68 admin admin] PATH_INFO: `/v3/projects/c66f88d2acc24b2198099c12f463e0c9` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:48.037027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a520e-813b-461a-a1a9-804320d4ce68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c66f88d2acc24b2198099c12f463e0c9)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:48.038939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a520e-813b-461a-a1a9-804320d4ce68 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:48.126389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b12a520e-813b-461a-a1a9-804320d4ce68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c66f88d2acc24b2198099c12f463e0c9', 'request_id': 'req-b12a520e-813b-461a-a1a9-804320d4ce68'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:48.132787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b12a520e-813b-461a-a1a9-804320d4ce68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c66f88d2acc24b2198099c12f463e0c9', 'request_id': 'req-b12a520e-813b-461a-a1a9-804320d4ce68'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:48.138252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b12a520e-813b-461a-a1a9-804320d4ce68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c66f88d2acc24b2198099c12f463e0c9', 'request_id': 'req-b12a520e-813b-461a-a1a9-804320d4ce68'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:48.221668 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b12a520e-813b-461a-a1a9-804320d4ce68 admin admin] The token cache is being invalidate because project c66f88d2acc24b2198099c12f463e0c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:59:48.221871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b12a520e-813b-461a-a1a9-804320d4ce68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:48.222500 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b12a520e-813b-461a-a1a9-804320d4ce68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:59:48.223993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1318/2610] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 16:59:47 2022] DELETE /identity/v3/projects/c66f88d2acc24b2198099c12f463e0c9 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:59:48.229138 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8d86e-d0e4-4035-ba6e-fe2ecf850787 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:48.229414 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8d86e-d0e4-4035-ba6e-fe2ecf850787 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:48.230717 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8d86e-d0e4-4035-ba6e-fe2ecf850787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:48.250030 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-11b8d86e-d0e4-4035-ba6e-fe2ecf850787 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:59:48.251218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-11b8d86e-d0e4-4035-ba6e-fe2ecf850787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:48.310968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1293/2611] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:59:48.318636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93160eb5-32c5-4295-a7a6-fa5aa42ce231 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:48.318924 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93160eb5-32c5-4295-a7a6-fa5aa42ce231 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:48.319202 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93160eb5-32c5-4295-a7a6-fa5aa42ce231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:48.339012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-93160eb5-32c5-4295-a7a6-fa5aa42ce231 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:59:48.340217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-93160eb5-32c5-4295-a7a6-fa5aa42ce231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:48.348108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1319/2612] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:59:48.352261 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8d86e-d0e4-4035-ba6e-fe2ecf850787 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:48.362205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8d86e-d0e4-4035-ba6e-fe2ecf850787 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:48.366664 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a33f5d-11de-4fad-8664-951314b29bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:48.367249 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a33f5d-11de-4fad-8664-951314b29bbc admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:48.367525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a33f5d-11de-4fad-8664-951314b29bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:48.367793 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a33f5d-11de-4fad-8664-951314b29bbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:48.368451 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a33f5d-11de-4fad-8664-951314b29bbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:59:48.368754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a33f5d-11de-4fad-8664-951314b29bbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:48.370360 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a33f5d-11de-4fad-8664-951314b29bbc admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:48.382307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1294/2613] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 16:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:59:48.386376 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93160eb5-32c5-4295-a7a6-fa5aa42ce231 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:48.395482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93160eb5-32c5-4295-a7a6-fa5aa42ce231 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:48.400164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812e6c7-64bc-4e2f-b1fc-90ab6cbc6e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:48.400796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812e6c7-64bc-4e2f-b1fc-90ab6cbc6e66 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:48.401100 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812e6c7-64bc-4e2f-b1fc-90ab6cbc6e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:48.401377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812e6c7-64bc-4e2f-b1fc-90ab6cbc6e66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:48.402043 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812e6c7-64bc-4e2f-b1fc-90ab6cbc6e66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:48.403955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812e6c7-64bc-4e2f-b1fc-90ab6cbc6e66 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:48.491286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1320/2614] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:59:48 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:59:48.495922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a33f5d-11de-4fad-8664-951314b29bbc admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:48.506846 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a33f5d-11de-4fad-8664-951314b29bbc admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:48.537816 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:59:48.562312 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e455edd-0e7a-4f2d-a6b8-a3bd02779d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:48.562919 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e455edd-0e7a-4f2d-a6b8-a3bd02779d21 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:48.563193 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e455edd-0e7a-4f2d-a6b8-a3bd02779d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:48.563460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e455edd-0e7a-4f2d-a6b8-a3bd02779d21 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:48.564071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e455edd-0e7a-4f2d-a6b8-a3bd02779d21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:48.566020 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e455edd-0e7a-4f2d-a6b8-a3bd02779d21 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:48.684499 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1295/2615] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:59:48 2022] POST /identity/v3/users => generated 283 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:59:48.689218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812e6c7-64bc-4e2f-b1fc-90ab6cbc6e66 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:48.699180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812e6c7-64bc-4e2f-b1fc-90ab6cbc6e66 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:48.703691 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38f35b-e3b1-46e8-a317-66b76753b877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:48.704276 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f35b-e3b1-46e8-a317-66b76753b877 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:48.704587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f35b-e3b1-46e8-a317-66b76753b877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:48.704861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f35b-e3b1-46e8-a317-66b76753b877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:48.705502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38f35b-e3b1-46e8-a317-66b76753b877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:48.707074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38f35b-e3b1-46e8-a317-66b76753b877 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:48.716930 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1321/2616] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:59:48.720957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e455edd-0e7a-4f2d-a6b8-a3bd02779d21 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:48.731153 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e455edd-0e7a-4f2d-a6b8-a3bd02779d21 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:48.736519 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1341e-d417-4393-b091-504bc5e3b7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:48.737121 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1341e-d417-4393-b091-504bc5e3b7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:48.739289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1341e-d417-4393-b091-504bc5e3b7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:48.739289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1341e-d417-4393-b091-504bc5e3b7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:48.739289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1341e-d417-4393-b091-504bc5e3b7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:48.739895 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1341e-d417-4393-b091-504bc5e3b7f0 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:48.751056 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1296/2617] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:59:48.755071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38f35b-e3b1-46e8-a317-66b76753b877 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:48.765183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38f35b-e3b1-46e8-a317-66b76753b877 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:48.769788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed4fa1-7277-4436-b5b5-039f3bcc1aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:48.770447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed4fa1-7277-4436-b5b5-039f3bcc1aae admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:48.770727 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed4fa1-7277-4436-b5b5-039f3bcc1aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:48.771004 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed4fa1-7277-4436-b5b5-039f3bcc1aae admin admin] PATH_INFO: `/v3/projects/0e18e10a84af41d4ad84cd51a07af5c3/users/6d841d59c4e2473baa72a8dd1c3ee17f/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:48.785887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed4fa1-7277-4436-b5b5-039f3bcc1aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e18e10a84af41d4ad84cd51a07af5c3, user_id=6d841d59c4e2473baa72a8dd1c3ee17f, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:48.788661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed4fa1-7277-4436-b5b5-039f3bcc1aae admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:48.908797 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1322/2618] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:59:48 2022] PUT /identity/v3/projects/0e18e10a84af41d4ad84cd51a07af5c3/users/6d841d59c4e2473baa72a8dd1c3ee17f/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:59:48.920877 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f358035-b2fb-4ec5-8073-0f7536b5544e None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:48.921161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f358035-b2fb-4ec5-8073-0f7536b5544e None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:48.921436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f358035-b2fb-4ec5-8073-0f7536b5544e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:48.949415 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-8f358035-b2fb-4ec5-8073-0f7536b5544e None None] MFA Rules not processed for user `6d841d59c4e2473baa72a8dd1c3ee17f`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:59:48.950632 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-8f358035-b2fb-4ec5-8073-0f7536b5544e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:49.014786 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1297/2619] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:59:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:59:49.019289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed4fa1-7277-4436-b5b5-039f3bcc1aae admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:49.079792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed4fa1-7277-4436-b5b5-039f3bcc1aae admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:49.084536 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d1b4e-e02b-451b-9656-551aba20b939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:49.085134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d1b4e-e02b-451b-9656-551aba20b939 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:49.085417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d1b4e-e02b-451b-9656-551aba20b939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:49.085693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d1b4e-e02b-451b-9656-551aba20b939 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:49.086327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d1b4e-e02b-451b-9656-551aba20b939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:49.088219 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d1b4e-e02b-451b-9656-551aba20b939 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:49.093323 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:59:49.167259 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1323/2620] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 16:59:49 2022] POST /identity/v3/projects => generated 365 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:59:49.171501 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f358035-b2fb-4ec5-8073-0f7536b5544e None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:49.182339 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f358035-b2fb-4ec5-8073-0f7536b5544e None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:49.238094 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cec955-18b5-4ea1-b099-3ae16a77e594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:49.238720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cec955-18b5-4ea1-b099-3ae16a77e594 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:49.238998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cec955-18b5-4ea1-b099-3ae16a77e594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:49.239266 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cec955-18b5-4ea1-b099-3ae16a77e594 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:49.239879 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cec955-18b5-4ea1-b099-3ae16a77e594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:49.241812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cec955-18b5-4ea1-b099-3ae16a77e594 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:49.345703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1298/2621] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 16:59:49 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:59:49.346823 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d1b4e-e02b-451b-9656-551aba20b939 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:49.358020 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d1b4e-e02b-451b-9656-551aba20b939 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:49.362756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31624c9-7d84-46da-b1e7-341db5744e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:49.363497 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31624c9-7d84-46da-b1e7-341db5744e09 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:49.363782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31624c9-7d84-46da-b1e7-341db5744e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:49.364057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31624c9-7d84-46da-b1e7-341db5744e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:49.364737 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31624c9-7d84-46da-b1e7-341db5744e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:49.366321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31624c9-7d84-46da-b1e7-341db5744e09 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:49.383794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1324/2622] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:59:49.385173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cec955-18b5-4ea1-b099-3ae16a77e594 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:49.396773 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cec955-18b5-4ea1-b099-3ae16a77e594 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:49.400056 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8fb56-d925-4593-8cb8-8290fa4542f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:49.400732 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8fb56-d925-4593-8cb8-8290fa4542f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:49.401008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8fb56-d925-4593-8cb8-8290fa4542f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:49.401275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8fb56-d925-4593-8cb8-8290fa4542f9 admin admin] PATH_INFO: `/v3/projects/82d5a48a981a4bfbaa371b74f9be4be8/users/7e0a89e5988c482394323a69dbd056fc/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:49.417223 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8fb56-d925-4593-8cb8-8290fa4542f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82d5a48a981a4bfbaa371b74f9be4be8, user_id=7e0a89e5988c482394323a69dbd056fc, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:49.420472 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8fb56-d925-4593-8cb8-8290fa4542f9 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:49.486875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1299/2623] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:59:49 2022] PUT /identity/v3/projects/82d5a48a981a4bfbaa371b74f9be4be8/users/7e0a89e5988c482394323a69dbd056fc/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:59:49.490896 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31624c9-7d84-46da-b1e7-341db5744e09 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:49.501096 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31624c9-7d84-46da-b1e7-341db5744e09 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:49.581167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec5d2f-02d2-46df-8120-20df2c9c576d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:49.583447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec5d2f-02d2-46df-8120-20df2c9c576d admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:49.583447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec5d2f-02d2-46df-8120-20df2c9c576d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:49.583447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec5d2f-02d2-46df-8120-20df2c9c576d admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:49.583447 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec5d2f-02d2-46df-8120-20df2c9c576d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:49.584844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec5d2f-02d2-46df-8120-20df2c9c576d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:49.595977 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1325/2624] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 16:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:59:49.600953 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8fb56-d925-4593-8cb8-8290fa4542f9 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:49.612255 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8fb56-d925-4593-8cb8-8290fa4542f9 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:49.616907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3776ba-ea2c-4bb8-bc7c-f5e853e50000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:49.617569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3776ba-ea2c-4bb8-bc7c-f5e853e50000 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:49.617847 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3776ba-ea2c-4bb8-bc7c-f5e853e50000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:49.618119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3776ba-ea2c-4bb8-bc7c-f5e853e50000 admin admin] PATH_INFO: `/v3/projects/82d5a48a981a4bfbaa371b74f9be4be8/users/7e0a89e5988c482394323a69dbd056fc/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:49.620021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3776ba-ea2c-4bb8-bc7c-f5e853e50000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82d5a48a981a4bfbaa371b74f9be4be8, user_id=7e0a89e5988c482394323a69dbd056fc, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:49.623309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3776ba-ea2c-4bb8-bc7c-f5e853e50000 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:49.626554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:59:49.634236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1300/2625] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 16:59:49 2022] PUT /identity/v3/projects/82d5a48a981a4bfbaa371b74f9be4be8/users/7e0a89e5988c482394323a69dbd056fc/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:59:49.644731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939d1a5-c567-4305-af96-879a1b0121a6 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:49.645017 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939d1a5-c567-4305-af96-879a1b0121a6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:49.645295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939d1a5-c567-4305-af96-879a1b0121a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:49.675027 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f939d1a5-c567-4305-af96-879a1b0121a6 None None] MFA Rules not processed for user `7e0a89e5988c482394323a69dbd056fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:59:49.676234 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f939d1a5-c567-4305-af96-879a1b0121a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:49.757473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1326/2626] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:59:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:59:49.769274 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3776ba-ea2c-4bb8-bc7c-f5e853e50000 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:49.771579 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-bc3776ba-ea2c-4bb8-bc7c-f5e853e50000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:49.829924 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3776ba-ea2c-4bb8-bc7c-f5e853e50000 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:49.833612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ea172-1d4b-4642-8d6d-6174ad3caad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:49.834219 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ea172-1d4b-4642-8d6d-6174ad3caad8 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:49.834565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ea172-1d4b-4642-8d6d-6174ad3caad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:49.834841 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ea172-1d4b-4642-8d6d-6174ad3caad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:49.844714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ea172-1d4b-4642-8d6d-6174ad3caad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:49.846195 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ea172-1d4b-4642-8d6d-6174ad3caad8 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:49.855809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1301/2627] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:59:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:59:50.047847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939d1a5-c567-4305-af96-879a1b0121a6 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:50.057407 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939d1a5-c567-4305-af96-879a1b0121a6 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:50.061516 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15780694-2c5f-44ff-b8ef-9995e409ca24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:50.062106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15780694-2c5f-44ff-b8ef-9995e409ca24 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:50.062382 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15780694-2c5f-44ff-b8ef-9995e409ca24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:50.062648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15780694-2c5f-44ff-b8ef-9995e409ca24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:50.115373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:59:50.119036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15780694-2c5f-44ff-b8ef-9995e409ca24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:50.120520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15780694-2c5f-44ff-b8ef-9995e409ca24 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:50.129726 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1327/2628] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 16:59:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:59:56.285497 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963f683-3b01-4e21-afce-00c221766103 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:56.285955 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963f683-3b01-4e21-afce-00c221766103 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:56.286081 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963f683-3b01-4e21-afce-00c221766103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:56.305673 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-2963f683-3b01-4e21-afce-00c221766103 None None] MFA Rules not processed for user `d715cc7828f847529f795f356a8ac16b`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:59:56.306846 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2963f683-3b01-4e21-afce-00c221766103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:56.484715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1302/2629] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 16:59:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:00.165677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:04.791709 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15780694-2c5f-44ff-b8ef-9995e409ca24 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:04.793134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-15780694-2c5f-44ff-b8ef-9995e409ca24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:04.890887 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15780694-2c5f-44ff-b8ef-9995e409ca24 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:04.904006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0d198-5bd9-442b-bd56-c806e06afdb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:04.904607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0d198-5bd9-442b-bd56-c806e06afdb1 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:04.905001 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0d198-5bd9-442b-bd56-c806e06afdb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:04.905378 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0d198-5bd9-442b-bd56-c806e06afdb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:04.907419 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-75f0d198-5bd9-442b-bd56-c806e06afdb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:04.922003 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0d198-5bd9-442b-bd56-c806e06afdb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:04.923235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0d198-5bd9-442b-bd56-c806e06afdb1 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:04.942013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1328/2630] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:00:06.476345 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=53 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:07.312664 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963f683-3b01-4e21-afce-00c221766103 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:07.374772 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963f683-3b01-4e21-afce-00c221766103 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:07.382805 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795499e0-4974-4102-b470-1084d9db8ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:07.385521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795499e0-4974-4102-b470-1084d9db8ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:07.385521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795499e0-4974-4102-b470-1084d9db8ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:07.385521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795499e0-4974-4102-b470-1084d9db8ec5 admin admin] PATH_INFO: `/v3/users/6d841d59c4e2473baa72a8dd1c3ee17f` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:07.385943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795499e0-4974-4102-b470-1084d9db8ec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d841d59c4e2473baa72a8dd1c3ee17f)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:07.387851 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795499e0-4974-4102-b470-1084d9db8ec5 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:07.621334 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-795499e0-4974-4102-b470-1084d9db8ec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d841d59c4e2473baa72a8dd1c3ee17f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:07.623324 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-795499e0-4974-4102-b470-1084d9db8ec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d841d59c4e2473baa72a8dd1c3ee17f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:07.623951 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-795499e0-4974-4102-b470-1084d9db8ec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d841d59c4e2473baa72a8dd1c3ee17f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:07.631961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-795499e0-4974-4102-b470-1084d9db8ec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d841d59c4e2473baa72a8dd1c3ee17f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:07.702276 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-795499e0-4974-4102-b470-1084d9db8ec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d841d59c4e2473baa72a8dd1c3ee17f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:07.720961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1303/2631] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:00:07 2022] DELETE /identity/v3/users/6d841d59c4e2473baa72a8dd1c3ee17f => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:07.725418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0567a9-5a29-4f72-ad25-2388add8cd12 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:07.725881 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0567a9-5a29-4f72-ad25-2388add8cd12 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:07.726024 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0567a9-5a29-4f72-ad25-2388add8cd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:07.744697 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ba0567a9-5a29-4f72-ad25-2388add8cd12 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:07.745892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ba0567a9-5a29-4f72-ad25-2388add8cd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:07.805205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1329/2632] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:07.817675 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795499e0-4974-4102-b470-1084d9db8ec5 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:07.819103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-795499e0-4974-4102-b470-1084d9db8ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:07.880631 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795499e0-4974-4102-b470-1084d9db8ec5 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:07.889851 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086edc5c-37da-48fe-afe7-c78ef10d9983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:07.890730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086edc5c-37da-48fe-afe7-c78ef10d9983 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:07.891017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086edc5c-37da-48fe-afe7-c78ef10d9983 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:07.891344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086edc5c-37da-48fe-afe7-c78ef10d9983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:07.901035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086edc5c-37da-48fe-afe7-c78ef10d9983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:07.902501 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086edc5c-37da-48fe-afe7-c78ef10d9983 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:07.912531 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1304/2633] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:00:08.368580 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0567a9-5a29-4f72-ad25-2388add8cd12 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:08.368580 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ba0567a9-5a29-4f72-ad25-2388add8cd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:08.379731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0567a9-5a29-4f72-ad25-2388add8cd12 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:08.384832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0379e0c-10cf-4068-a195-f47a22f36974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:08.385471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0379e0c-10cf-4068-a195-f47a22f36974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:08.385753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0379e0c-10cf-4068-a195-f47a22f36974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:08.386071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0379e0c-10cf-4068-a195-f47a22f36974 admin admin] PATH_INFO: `/v3/projects/0e18e10a84af41d4ad84cd51a07af5c3` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:08.387282 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0379e0c-10cf-4068-a195-f47a22f36974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e18e10a84af41d4ad84cd51a07af5c3)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:08.389888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0379e0c-10cf-4068-a195-f47a22f36974 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:08.500695 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0379e0c-10cf-4068-a195-f47a22f36974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e18e10a84af41d4ad84cd51a07af5c3', 'request_id': 'req-a0379e0c-10cf-4068-a195-f47a22f36974'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:08.508319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0379e0c-10cf-4068-a195-f47a22f36974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e18e10a84af41d4ad84cd51a07af5c3', 'request_id': 'req-a0379e0c-10cf-4068-a195-f47a22f36974'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:08.514534 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0379e0c-10cf-4068-a195-f47a22f36974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e18e10a84af41d4ad84cd51a07af5c3', 'request_id': 'req-a0379e0c-10cf-4068-a195-f47a22f36974'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:08.605185 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0379e0c-10cf-4068-a195-f47a22f36974 admin admin] The token cache is being invalidate because project 0e18e10a84af41d4ad84cd51a07af5c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:00:08.605480 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0379e0c-10cf-4068-a195-f47a22f36974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:08.609308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0379e0c-10cf-4068-a195-f47a22f36974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:08.609308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1330/2634] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:00:08 2022] DELETE /identity/v3/projects/0e18e10a84af41d4ad84cd51a07af5c3 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:08.613751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086edc5c-37da-48fe-afe7-c78ef10d9983 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:08.623842 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086edc5c-37da-48fe-afe7-c78ef10d9983 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:08.643409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:08.681545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6606b03-d727-47bc-96ed-0c6fb4e7e067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:08.682176 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6606b03-d727-47bc-96ed-0c6fb4e7e067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:08.682476 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6606b03-d727-47bc-96ed-0c6fb4e7e067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:08.682756 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6606b03-d727-47bc-96ed-0c6fb4e7e067 admin admin] PATH_INFO: `/v3/users/7e0a89e5988c482394323a69dbd056fc` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:08.684014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6606b03-d727-47bc-96ed-0c6fb4e7e067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e0a89e5988c482394323a69dbd056fc)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:08.686085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6606b03-d727-47bc-96ed-0c6fb4e7e067 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:08.813371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a6606b03-d727-47bc-96ed-0c6fb4e7e067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e0a89e5988c482394323a69dbd056fc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:08.814179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a6606b03-d727-47bc-96ed-0c6fb4e7e067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e0a89e5988c482394323a69dbd056fc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:08.814797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a6606b03-d727-47bc-96ed-0c6fb4e7e067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e0a89e5988c482394323a69dbd056fc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:08.821971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a6606b03-d727-47bc-96ed-0c6fb4e7e067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e0a89e5988c482394323a69dbd056fc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:08.874161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-a6606b03-d727-47bc-96ed-0c6fb4e7e067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e0a89e5988c482394323a69dbd056fc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:08.899346 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1305/2635] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:00:08 2022] DELETE /identity/v3/users/7e0a89e5988c482394323a69dbd056fc => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:09.367438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0379e0c-10cf-4068-a195-f47a22f36974 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:09.369588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a0379e0c-10cf-4068-a195-f47a22f36974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:09.386782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0379e0c-10cf-4068-a195-f47a22f36974 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:09.392743 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:09.445466 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76371d5-e0eb-4a1b-93c9-d34b761f727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:09.446106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76371d5-e0eb-4a1b-93c9-d34b761f727b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:09.446388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76371d5-e0eb-4a1b-93c9-d34b761f727b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:09.446664 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76371d5-e0eb-4a1b-93c9-d34b761f727b admin admin] PATH_INFO: `/v3/projects/82d5a48a981a4bfbaa371b74f9be4be8` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:09.447806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76371d5-e0eb-4a1b-93c9-d34b761f727b admin admin] RBAC: Authorizing `identity:delete_project(project_id=82d5a48a981a4bfbaa371b74f9be4be8)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:09.449819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76371d5-e0eb-4a1b-93c9-d34b761f727b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:09.586049 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b76371d5-e0eb-4a1b-93c9-d34b761f727b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82d5a48a981a4bfbaa371b74f9be4be8', 'request_id': 'req-b76371d5-e0eb-4a1b-93c9-d34b761f727b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:09.592446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b76371d5-e0eb-4a1b-93c9-d34b761f727b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82d5a48a981a4bfbaa371b74f9be4be8', 'request_id': 'req-b76371d5-e0eb-4a1b-93c9-d34b761f727b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:09.599947 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b76371d5-e0eb-4a1b-93c9-d34b761f727b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82d5a48a981a4bfbaa371b74f9be4be8', 'request_id': 'req-b76371d5-e0eb-4a1b-93c9-d34b761f727b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:09.704035 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b76371d5-e0eb-4a1b-93c9-d34b761f727b admin admin] The token cache is being invalidate because project 82d5a48a981a4bfbaa371b74f9be4be8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:00:09.704329 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b76371d5-e0eb-4a1b-93c9-d34b761f727b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:09.704995 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b76371d5-e0eb-4a1b-93c9-d34b761f727b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:09.707077 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1331/2636] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:00:09 2022] DELETE /identity/v3/projects/82d5a48a981a4bfbaa371b74f9be4be8 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:09.711682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754bda9-6021-430b-a9b2-3e575c1a15e0 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:09.712195 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754bda9-6021-430b-a9b2-3e575c1a15e0 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:09.712308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754bda9-6021-430b-a9b2-3e575c1a15e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:09.733041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-6754bda9-6021-430b-a9b2-3e575c1a15e0 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:09.734251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6754bda9-6021-430b-a9b2-3e575c1a15e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:09.799260 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1306/2637] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:09.807114 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47365101-18ec-40a9-bfcb-9544b3e48be8 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:09.807423 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47365101-18ec-40a9-bfcb-9544b3e48be8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:09.807701 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47365101-18ec-40a9-bfcb-9544b3e48be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:09.828069 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-47365101-18ec-40a9-bfcb-9544b3e48be8 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:09.829313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-47365101-18ec-40a9-bfcb-9544b3e48be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:09.838598 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1332/2638] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:09.842884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754bda9-6021-430b-a9b2-3e575c1a15e0 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:09.852693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754bda9-6021-430b-a9b2-3e575c1a15e0 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:09.857918 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e932c07-2d4a-4def-84c2-001ae5882112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:09.858528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e932c07-2d4a-4def-84c2-001ae5882112 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:09.858808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e932c07-2d4a-4def-84c2-001ae5882112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:09.859080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e932c07-2d4a-4def-84c2-001ae5882112 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:09.859758 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e932c07-2d4a-4def-84c2-001ae5882112 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:00:09.860060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e932c07-2d4a-4def-84c2-001ae5882112 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:09.861683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e932c07-2d4a-4def-84c2-001ae5882112 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:09.874117 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1307/2639] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 17:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:00:09.878547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47365101-18ec-40a9-bfcb-9544b3e48be8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:09.888064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47365101-18ec-40a9-bfcb-9544b3e48be8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:09.893988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9244d71-06ee-46c7-acdd-33c3abcdda58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:09.894604 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9244d71-06ee-46c7-acdd-33c3abcdda58 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:09.894888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9244d71-06ee-46c7-acdd-33c3abcdda58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:09.895675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9244d71-06ee-46c7-acdd-33c3abcdda58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:09.896354 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9244d71-06ee-46c7-acdd-33c3abcdda58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:09.901571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9244d71-06ee-46c7-acdd-33c3abcdda58 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:09.966837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1333/2640] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:00:09 2022] POST /identity/v3/projects => generated 387 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:00:09.970768 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e932c07-2d4a-4def-84c2-001ae5882112 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:09.982065 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e932c07-2d4a-4def-84c2-001ae5882112 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:10.008167 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:10.038494 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea40335-53eb-4844-acdb-8fad730c4a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:10.039095 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea40335-53eb-4844-acdb-8fad730c4a7f admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:10.039374 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea40335-53eb-4844-acdb-8fad730c4a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:10.039645 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea40335-53eb-4844-acdb-8fad730c4a7f admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:10.040364 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea40335-53eb-4844-acdb-8fad730c4a7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:10.042292 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea40335-53eb-4844-acdb-8fad730c4a7f admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:10.096528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1308/2641] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:00:09 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:00:10.100657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9244d71-06ee-46c7-acdd-33c3abcdda58 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:10.111333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9244d71-06ee-46c7-acdd-33c3abcdda58 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:10.116034 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebe6e3-da7c-4946-b013-446964931672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:10.116669 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebe6e3-da7c-4946-b013-446964931672 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:10.116952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebe6e3-da7c-4946-b013-446964931672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:10.117223 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebe6e3-da7c-4946-b013-446964931672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:10.117863 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebe6e3-da7c-4946-b013-446964931672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:10.119467 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebe6e3-da7c-4946-b013-446964931672 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:10.129446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1334/2642] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:00:10.133674 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea40335-53eb-4844-acdb-8fad730c4a7f admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:10.143692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea40335-53eb-4844-acdb-8fad730c4a7f admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:10.148282 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52446668-b9c0-4cdf-a857-4ad4801a5fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:10.148913 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52446668-b9c0-4cdf-a857-4ad4801a5fde admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:10.149296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52446668-b9c0-4cdf-a857-4ad4801a5fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:10.149576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52446668-b9c0-4cdf-a857-4ad4801a5fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:10.150214 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52446668-b9c0-4cdf-a857-4ad4801a5fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:10.151788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52446668-b9c0-4cdf-a857-4ad4801a5fde admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:10.162408 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1309/2643] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:00:10.166708 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebe6e3-da7c-4946-b013-446964931672 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:10.177283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebe6e3-da7c-4946-b013-446964931672 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:10.181979 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305aee2f-e028-44e4-91f9-35b5f118026b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:10.182634 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305aee2f-e028-44e4-91f9-35b5f118026b admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:10.182915 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305aee2f-e028-44e4-91f9-35b5f118026b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:10.183190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305aee2f-e028-44e4-91f9-35b5f118026b admin admin] PATH_INFO: `/v3/projects/cb663b86bbae4e24bc07c50e8190bb05/users/885c1bb9d53d45bd86cc1a5b5c34ae98/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:10.318789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305aee2f-e028-44e4-91f9-35b5f118026b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb663b86bbae4e24bc07c50e8190bb05, user_id=885c1bb9d53d45bd86cc1a5b5c34ae98, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:10.321543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305aee2f-e028-44e4-91f9-35b5f118026b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:10.343939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52446668-b9c0-4cdf-a857-4ad4801a5fde admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:10.343939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-52446668-b9c0-4cdf-a857-4ad4801a5fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:10.383258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1335/2644] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:00:10 2022] PUT /identity/v3/projects/cb663b86bbae4e24bc07c50e8190bb05/users/885c1bb9d53d45bd86cc1a5b5c34ae98/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:10.396972 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095ac48-6b62-4af4-a02f-844a2b93c31c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:10.397321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095ac48-6b62-4af4-a02f-844a2b93c31c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:10.397599 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095ac48-6b62-4af4-a02f-844a2b93c31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:10.399229 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52446668-b9c0-4cdf-a857-4ad4801a5fde admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:10.424385 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5095ac48-6b62-4af4-a02f-844a2b93c31c None None] MFA Rules not processed for user `885c1bb9d53d45bd86cc1a5b5c34ae98`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:10.425620 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5095ac48-6b62-4af4-a02f-844a2b93c31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:10.451348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06644840-572c-47a1-8997-24bce8bfde6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:10.451591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:10.451838 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06644840-572c-47a1-8997-24bce8bfde6c service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:10.452118 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06644840-572c-47a1-8997-24bce8bfde6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:10.452417 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06644840-572c-47a1-8997-24bce8bfde6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:10.454249 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-06644840-572c-47a1-8997-24bce8bfde6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:10.493412 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1336/2645] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:00:10.499417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095ac48-6b62-4af4-a02f-844a2b93c31c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:10.525971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06644840-572c-47a1-8997-24bce8bfde6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:10.527410 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06644840-572c-47a1-8997-24bce8bfde6c service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:10.539537 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1310/2646] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:00:10.572338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095ac48-6b62-4af4-a02f-844a2b93c31c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:10.577732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13a272-43b4-4a8e-88ec-93e9f3888640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:10.578342 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13a272-43b4-4a8e-88ec-93e9f3888640 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:10.578752 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13a272-43b4-4a8e-88ec-93e9f3888640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:10.579130 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13a272-43b4-4a8e-88ec-93e9f3888640 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:10.579766 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13a272-43b4-4a8e-88ec-93e9f3888640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:10.581703 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13a272-43b4-4a8e-88ec-93e9f3888640 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:10.676057 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1337/2647] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:00:10 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:00:10.680591 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06644840-572c-47a1-8997-24bce8bfde6c service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:10.690231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06644840-572c-47a1-8997-24bce8bfde6c service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:10.699085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:10.750395 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6672c33-f4d0-4969-9163-86f2851354dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:10.751045 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6672c33-f4d0-4969-9163-86f2851354dc admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:10.751219 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6672c33-f4d0-4969-9163-86f2851354dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:10.751599 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6672c33-f4d0-4969-9163-86f2851354dc admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:10.752233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6672c33-f4d0-4969-9163-86f2851354dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:10.754218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6672c33-f4d0-4969-9163-86f2851354dc admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:10.881789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1311/2648] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:00:10 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:00:10.886451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13a272-43b4-4a8e-88ec-93e9f3888640 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:10.898804 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13a272-43b4-4a8e-88ec-93e9f3888640 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:10.905359 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9d109-f821-47fa-9f7c-bfdae79239ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:10.906075 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9d109-f821-47fa-9f7c-bfdae79239ff admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:10.908094 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9d109-f821-47fa-9f7c-bfdae79239ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:10.908094 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9d109-f821-47fa-9f7c-bfdae79239ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:10.908094 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9d109-f821-47fa-9f7c-bfdae79239ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:10.912167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9d109-f821-47fa-9f7c-bfdae79239ff admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:10.922335 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1338/2649] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:00:10.926635 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6672c33-f4d0-4969-9163-86f2851354dc admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:10.939852 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6672c33-f4d0-4969-9163-86f2851354dc admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:10.947798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca95186-b02b-4f3f-b139-aa5eb3144423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:10.949016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca95186-b02b-4f3f-b139-aa5eb3144423 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:10.949467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca95186-b02b-4f3f-b139-aa5eb3144423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:10.950372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca95186-b02b-4f3f-b139-aa5eb3144423 admin admin] PATH_INFO: `/v3/projects/52f2d2f45d764ad59d5bbd9a195c3b51/users/a9163895599a4c76b08dd09d79b2ebc3/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:10.968995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca95186-b02b-4f3f-b139-aa5eb3144423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52f2d2f45d764ad59d5bbd9a195c3b51, user_id=a9163895599a4c76b08dd09d79b2ebc3, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:10.971704 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca95186-b02b-4f3f-b139-aa5eb3144423 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:11.072115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1312/2650] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:00:10 2022] PUT /identity/v3/projects/52f2d2f45d764ad59d5bbd9a195c3b51/users/a9163895599a4c76b08dd09d79b2ebc3/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:11.076125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9d109-f821-47fa-9f7c-bfdae79239ff admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:11.089096 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9d109-f821-47fa-9f7c-bfdae79239ff admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:11.158352 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eae55f-c29d-4659-a3e3-d7ec71ae8b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:11.158969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eae55f-c29d-4659-a3e3-d7ec71ae8b9b admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:11.159253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eae55f-c29d-4659-a3e3-d7ec71ae8b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:11.159559 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eae55f-c29d-4659-a3e3-d7ec71ae8b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:11.160275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eae55f-c29d-4659-a3e3-d7ec71ae8b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:11.161925 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eae55f-c29d-4659-a3e3-d7ec71ae8b9b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:11.179498 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1339/2651] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:00:11.182402 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca95186-b02b-4f3f-b139-aa5eb3144423 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:11.193566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca95186-b02b-4f3f-b139-aa5eb3144423 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:11.199631 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ca3ba-a167-4aa9-8bc4-5bca80b615c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:11.200380 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ca3ba-a167-4aa9-8bc4-5bca80b615c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:11.200805 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ca3ba-a167-4aa9-8bc4-5bca80b615c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:11.201073 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ca3ba-a167-4aa9-8bc4-5bca80b615c5 admin admin] PATH_INFO: `/v3/projects/52f2d2f45d764ad59d5bbd9a195c3b51/users/a9163895599a4c76b08dd09d79b2ebc3/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:11.202987 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ca3ba-a167-4aa9-8bc4-5bca80b615c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52f2d2f45d764ad59d5bbd9a195c3b51, user_id=a9163895599a4c76b08dd09d79b2ebc3, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:11.205643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ca3ba-a167-4aa9-8bc4-5bca80b615c5 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:11.218456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1313/2652] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:00:11 2022] PUT /identity/v3/projects/52f2d2f45d764ad59d5bbd9a195c3b51/users/a9163895599a4c76b08dd09d79b2ebc3/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:11.228819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101e1e8-7c4f-4442-aecd-804592897c8a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:11.228975 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101e1e8-7c4f-4442-aecd-804592897c8a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:11.229153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101e1e8-7c4f-4442-aecd-804592897c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:11.258698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9101e1e8-7c4f-4442-aecd-804592897c8a None None] MFA Rules not processed for user `a9163895599a4c76b08dd09d79b2ebc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:11.259899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9101e1e8-7c4f-4442-aecd-804592897c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:11.285468 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:11.332084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1340/2653] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:11.344526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ca3ba-a167-4aa9-8bc4-5bca80b615c5 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:11.346399 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c81ca3ba-a167-4aa9-8bc4-5bca80b615c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:11.408706 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ca3ba-a167-4aa9-8bc4-5bca80b615c5 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:11.412490 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7891c0-75e6-4b8a-9551-53c1b275c9f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:11.413105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7891c0-75e6-4b8a-9551-53c1b275c9f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:11.413387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7891c0-75e6-4b8a-9551-53c1b275c9f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:11.413658 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7891c0-75e6-4b8a-9551-53c1b275c9f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:11.424819 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7891c0-75e6-4b8a-9551-53c1b275c9f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:11.428832 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7891c0-75e6-4b8a-9551-53c1b275c9f0 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:11.437093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1314/2654] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:00:11.702920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101e1e8-7c4f-4442-aecd-804592897c8a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:11.784325 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101e1e8-7c4f-4442-aecd-804592897c8a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:11.789449 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edc587-95e8-4820-8c83-2bcb04b41393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:11.790085 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edc587-95e8-4820-8c83-2bcb04b41393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:11.790362 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edc587-95e8-4820-8c83-2bcb04b41393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:11.791561 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edc587-95e8-4820-8c83-2bcb04b41393 admin admin] PATH_INFO: `/v3/users/885c1bb9d53d45bd86cc1a5b5c34ae98` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:11.792128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edc587-95e8-4820-8c83-2bcb04b41393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=885c1bb9d53d45bd86cc1a5b5c34ae98)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:11.794922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edc587-95e8-4820-8c83-2bcb04b41393 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:11.956912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-99edc587-95e8-4820-8c83-2bcb04b41393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '885c1bb9d53d45bd86cc1a5b5c34ae98'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:11.957914 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-99edc587-95e8-4820-8c83-2bcb04b41393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '885c1bb9d53d45bd86cc1a5b5c34ae98'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:11.958626 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-99edc587-95e8-4820-8c83-2bcb04b41393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '885c1bb9d53d45bd86cc1a5b5c34ae98'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:12.039873 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-99edc587-95e8-4820-8c83-2bcb04b41393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '885c1bb9d53d45bd86cc1a5b5c34ae98'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:12.047430 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-99edc587-95e8-4820-8c83-2bcb04b41393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '885c1bb9d53d45bd86cc1a5b5c34ae98'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:12.064735 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1341/2655] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:00:11 2022] DELETE /identity/v3/users/885c1bb9d53d45bd86cc1a5b5c34ae98 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:12.069293 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96118f07-b97f-4fc1-85e6-7947c9069954 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:12.072738 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96118f07-b97f-4fc1-85e6-7947c9069954 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:12.072738 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96118f07-b97f-4fc1-85e6-7947c9069954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:12.089060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-96118f07-b97f-4fc1-85e6-7947c9069954 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:12.090269 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-96118f07-b97f-4fc1-85e6-7947c9069954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:12.164071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1315/2656] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:12.171007 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edc587-95e8-4820-8c83-2bcb04b41393 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:12.172945 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-99edc587-95e8-4820-8c83-2bcb04b41393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:12.240390 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:12.258838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edc587-95e8-4820-8c83-2bcb04b41393 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:12.276529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc186b-4e44-41e4-9662-905d1a05cc66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:12.277268 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc186b-4e44-41e4-9662-905d1a05cc66 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:12.277553 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc186b-4e44-41e4-9662-905d1a05cc66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:12.277827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc186b-4e44-41e4-9662-905d1a05cc66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:12.304038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc186b-4e44-41e4-9662-905d1a05cc66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:12.305720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc186b-4e44-41e4-9662-905d1a05cc66 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:12.319228 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1342/2657] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:00:12.842303 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96118f07-b97f-4fc1-85e6-7947c9069954 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:12.846710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-96118f07-b97f-4fc1-85e6-7947c9069954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:12.861024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96118f07-b97f-4fc1-85e6-7947c9069954 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:12.867258 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:12.867896 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:12.868264 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:12.868511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d admin admin] PATH_INFO: `/v3/projects/cb663b86bbae4e24bc07c50e8190bb05` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:12.869869 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb663b86bbae4e24bc07c50e8190bb05)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:12.872699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:12.875082 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:13.014432 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb663b86bbae4e24bc07c50e8190bb05', 'request_id': 'req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:13.021088 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb663b86bbae4e24bc07c50e8190bb05', 'request_id': 'req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:13.028077 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb663b86bbae4e24bc07c50e8190bb05', 'request_id': 'req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:13.113885 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d admin admin] The token cache is being invalidate because project cb663b86bbae4e24bc07c50e8190bb05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:00:13.114552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:13.115273 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:13.117053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1316/2658] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:00:12 2022] DELETE /identity/v3/projects/cb663b86bbae4e24bc07c50e8190bb05 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:13.121054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc186b-4e44-41e4-9662-905d1a05cc66 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:13.132373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc186b-4e44-41e4-9662-905d1a05cc66 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:13.223435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bcce1b-9913-4baa-b895-121a1eb5ce86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:13.223435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcce1b-9913-4baa-b895-121a1eb5ce86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:13.223435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcce1b-9913-4baa-b895-121a1eb5ce86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:13.223435 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcce1b-9913-4baa-b895-121a1eb5ce86 admin admin] PATH_INFO: `/v3/users/a9163895599a4c76b08dd09d79b2ebc3` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:13.224177 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bcce1b-9913-4baa-b895-121a1eb5ce86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9163895599a4c76b08dd09d79b2ebc3)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:13.227224 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bcce1b-9913-4baa-b895-121a1eb5ce86 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:13.440655 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-38bcce1b-9913-4baa-b895-121a1eb5ce86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9163895599a4c76b08dd09d79b2ebc3'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:13.441358 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-38bcce1b-9913-4baa-b895-121a1eb5ce86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9163895599a4c76b08dd09d79b2ebc3'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:13.441996 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-38bcce1b-9913-4baa-b895-121a1eb5ce86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9163895599a4c76b08dd09d79b2ebc3'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:13.520584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-38bcce1b-9913-4baa-b895-121a1eb5ce86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9163895599a4c76b08dd09d79b2ebc3'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:13.544528 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-38bcce1b-9913-4baa-b895-121a1eb5ce86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9163895599a4c76b08dd09d79b2ebc3'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:13.596531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1343/2659] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:00:13 2022] DELETE /identity/v3/users/a9163895599a4c76b08dd09d79b2ebc3 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:14.099893 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:14.099893 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:14.120417 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966aed6d-a660-4ff4-b6e9-2ec7f8e8161d admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:14.236893 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20b906-6af4-492f-80e5-073f9b923997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:14.237712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20b906-6af4-492f-80e5-073f9b923997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:14.237995 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20b906-6af4-492f-80e5-073f9b923997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:14.238271 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20b906-6af4-492f-80e5-073f9b923997 admin admin] PATH_INFO: `/v3/projects/52f2d2f45d764ad59d5bbd9a195c3b51` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:14.239441 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20b906-6af4-492f-80e5-073f9b923997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52f2d2f45d764ad59d5bbd9a195c3b51)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:14.243333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20b906-6af4-492f-80e5-073f9b923997 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:14.365162 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7c20b906-6af4-492f-80e5-073f9b923997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52f2d2f45d764ad59d5bbd9a195c3b51', 'request_id': 'req-7c20b906-6af4-492f-80e5-073f9b923997'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:14.378294 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7c20b906-6af4-492f-80e5-073f9b923997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52f2d2f45d764ad59d5bbd9a195c3b51', 'request_id': 'req-7c20b906-6af4-492f-80e5-073f9b923997'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:14.384950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7c20b906-6af4-492f-80e5-073f9b923997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52f2d2f45d764ad59d5bbd9a195c3b51', 'request_id': 'req-7c20b906-6af4-492f-80e5-073f9b923997'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:14.501627 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7c20b906-6af4-492f-80e5-073f9b923997 admin admin] The token cache is being invalidate because project 52f2d2f45d764ad59d5bbd9a195c3b51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:00:14.501627 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7c20b906-6af4-492f-80e5-073f9b923997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:14.501627 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-7c20b906-6af4-492f-80e5-073f9b923997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:14.501627 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1317/2660] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:00:14 2022] DELETE /identity/v3/projects/52f2d2f45d764ad59d5bbd9a195c3b51 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:14.502046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50b474-7e65-499f-b658-7a62f19648e3 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:14.502046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50b474-7e65-499f-b658-7a62f19648e3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:14.502046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50b474-7e65-499f-b658-7a62f19648e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:14.537218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9f50b474-7e65-499f-b658-7a62f19648e3 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:14.538584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9f50b474-7e65-499f-b658-7a62f19648e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:14.604583 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1344/2661] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:00:14.614217 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad010e-f107-48c2-b828-3661b886163f None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:14.614525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad010e-f107-48c2-b828-3661b886163f None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:14.615085 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad010e-f107-48c2-b828-3661b886163f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:14.638244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-40ad010e-f107-48c2-b828-3661b886163f None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:14.639693 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-40ad010e-f107-48c2-b828-3661b886163f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:14.648837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1318/2662] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:14.660164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50b474-7e65-499f-b658-7a62f19648e3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:14.665326 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:14.671032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50b474-7e65-499f-b658-7a62f19648e3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:14.675917 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0877e21-a0b3-4b0c-9b0b-21cfd09ebecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:14.676728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0877e21-a0b3-4b0c-9b0b-21cfd09ebecd admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:14.677015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0877e21-a0b3-4b0c-9b0b-21cfd09ebecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:14.677427 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0877e21-a0b3-4b0c-9b0b-21cfd09ebecd admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:14.677937 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0877e21-a0b3-4b0c-9b0b-21cfd09ebecd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:00:14.678258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0877e21-a0b3-4b0c-9b0b-21cfd09ebecd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:14.679886 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0877e21-a0b3-4b0c-9b0b-21cfd09ebecd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:14.694878 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1345/2663] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 17:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:00:14.697013 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad010e-f107-48c2-b828-3661b886163f None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:14.706639 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad010e-f107-48c2-b828-3661b886163f None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:14.711410 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb619b5b-4f41-43ae-b0a6-99e345872735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:14.712048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb619b5b-4f41-43ae-b0a6-99e345872735 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:14.726131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb619b5b-4f41-43ae-b0a6-99e345872735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:14.726131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb619b5b-4f41-43ae-b0a6-99e345872735 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:14.726131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb619b5b-4f41-43ae-b0a6-99e345872735 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:14.726131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb619b5b-4f41-43ae-b0a6-99e345872735 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:14.742899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:14.835936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1319/2664] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:00:14 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:00:14.840907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0877e21-a0b3-4b0c-9b0b-21cfd09ebecd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:14.850683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0877e21-a0b3-4b0c-9b0b-21cfd09ebecd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:14.896081 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb619b5b-4f41-43ae-b0a6-99e345872735 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:14.897867 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-bb619b5b-4f41-43ae-b0a6-99e345872735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:14.921547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdbf4a7-4309-42a8-ba60-4185a30206f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:14.922193 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbf4a7-4309-42a8-ba60-4185a30206f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:14.922475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbf4a7-4309-42a8-ba60-4185a30206f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:14.922774 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbf4a7-4309-42a8-ba60-4185a30206f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:14.923397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdbf4a7-4309-42a8-ba60-4185a30206f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:14.925372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdbf4a7-4309-42a8-ba60-4185a30206f1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:14.967319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb619b5b-4f41-43ae-b0a6-99e345872735 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:14.973066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8889e-ec02-42cf-905b-bd32a223f284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:14.973703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8889e-ec02-42cf-905b-bd32a223f284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:14.973983 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8889e-ec02-42cf-905b-bd32a223f284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:14.974256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8889e-ec02-42cf-905b-bd32a223f284 admin admin] PATH_INFO: `/v3/users/7a930257a6584b04bae278aa83940264` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:14.975391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8889e-ec02-42cf-905b-bd32a223f284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a930257a6584b04bae278aa83940264)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:14.977448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8889e-ec02-42cf-905b-bd32a223f284 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:14.995311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1346/2665] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:00:14 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:00:14.999528 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdbf4a7-4309-42a8-ba60-4185a30206f1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:15.009683 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdbf4a7-4309-42a8-ba60-4185a30206f1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:15.015009 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1f781-6d11-4f20-8194-16cf8782aace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:15.015602 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1f781-6d11-4f20-8194-16cf8782aace admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:15.015880 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1f781-6d11-4f20-8194-16cf8782aace admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:15.016150 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1f781-6d11-4f20-8194-16cf8782aace admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:15.016833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1f781-6d11-4f20-8194-16cf8782aace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:15.018413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1f781-6d11-4f20-8194-16cf8782aace admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:15.029403 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1347/2666] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:00:15.033426 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1f781-6d11-4f20-8194-16cf8782aace admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:15.042642 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1f781-6d11-4f20-8194-16cf8782aace admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:15.047278 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84547bff-e680-49fc-84a8-9c52962a19e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:15.047949 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84547bff-e680-49fc-84a8-9c52962a19e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:15.048233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84547bff-e680-49fc-84a8-9c52962a19e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:15.048531 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84547bff-e680-49fc-84a8-9c52962a19e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:15.049153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84547bff-e680-49fc-84a8-9c52962a19e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:15.050770 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84547bff-e680-49fc-84a8-9c52962a19e8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:15.060749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1348/2667] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:00:15.065015 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84547bff-e680-49fc-84a8-9c52962a19e8 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:15.074277 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84547bff-e680-49fc-84a8-9c52962a19e8 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:15.079373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d1b50-b156-40ea-b0c2-69f723f3bab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:15.079984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d1b50-b156-40ea-b0c2-69f723f3bab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:15.080262 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d1b50-b156-40ea-b0c2-69f723f3bab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:15.080564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d1b50-b156-40ea-b0c2-69f723f3bab5 admin admin] PATH_INFO: `/v3/projects/865a94937481403ebef955ea3dd821cd/users/d42f2da0912941bfb68a2691eac7b8c7/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:15.097947 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d1b50-b156-40ea-b0c2-69f723f3bab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=865a94937481403ebef955ea3dd821cd, user_id=d42f2da0912941bfb68a2691eac7b8c7, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:15.100991 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d1b50-b156-40ea-b0c2-69f723f3bab5 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:15.207333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2db8889e-ec02-42cf-905b-bd32a223f284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a930257a6584b04bae278aa83940264'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:15.208018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2db8889e-ec02-42cf-905b-bd32a223f284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a930257a6584b04bae278aa83940264'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:15.208713 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2db8889e-ec02-42cf-905b-bd32a223f284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a930257a6584b04bae278aa83940264'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:15.216590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2db8889e-ec02-42cf-905b-bd32a223f284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a930257a6584b04bae278aa83940264'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:15.223251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1349/2668] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:00:15 2022] PUT /identity/v3/projects/865a94937481403ebef955ea3dd821cd/users/d42f2da0912941bfb68a2691eac7b8c7/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:00:15.233703 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763ee11-47e9-4884-afbe-b4d1893367d7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:15.233934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763ee11-47e9-4884-afbe-b4d1893367d7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:15.234212 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763ee11-47e9-4884-afbe-b4d1893367d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:15.261994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5763ee11-47e9-4884-afbe-b4d1893367d7 None None] MFA Rules not processed for user `d42f2da0912941bfb68a2691eac7b8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:15.263704 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5763ee11-47e9-4884-afbe-b4d1893367d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:15.285234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2db8889e-ec02-42cf-905b-bd32a223f284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a930257a6584b04bae278aa83940264'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:15.304789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1320/2669] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:00:14 2022] DELETE /identity/v3/users/7a930257a6584b04bae278aa83940264 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:00:15.328518 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1350/2670] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:00:15.333081 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8889e-ec02-42cf-905b-bd32a223f284 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:15.334520 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2db8889e-ec02-42cf-905b-bd32a223f284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:15.400252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8889e-ec02-42cf-905b-bd32a223f284 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:15.405118 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d58098-570a-4f90-8d1f-6dea00731dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:15.405824 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d58098-570a-4f90-8d1f-6dea00731dff admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:15.406107 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d58098-570a-4f90-8d1f-6dea00731dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:15.406383 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d58098-570a-4f90-8d1f-6dea00731dff admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:15.407024 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d58098-570a-4f90-8d1f-6dea00731dff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:15.408972 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d58098-570a-4f90-8d1f-6dea00731dff admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:15.505396 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:15.513428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1321/2671] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:00:15 2022] POST /identity/v3/projects => generated 381 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:00:15.517529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763ee11-47e9-4884-afbe-b4d1893367d7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:15.545541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:15.585446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763ee11-47e9-4884-afbe-b4d1893367d7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:15.601864 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d58098-570a-4f90-8d1f-6dea00731dff admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:15.603619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-61d58098-570a-4f90-8d1f-6dea00731dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:15.662957 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f992b2-9e91-4fa6-9649-3ec92886f4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:15.663450 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f992b2-9e91-4fa6-9649-3ec92886f4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:15.663807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f992b2-9e91-4fa6-9649-3ec92886f4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:15.664153 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f992b2-9e91-4fa6-9649-3ec92886f4a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:15.664913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f992b2-9e91-4fa6-9649-3ec92886f4a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:15.667602 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f992b2-9e91-4fa6-9649-3ec92886f4a2 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:15.685911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d58098-570a-4f90-8d1f-6dea00731dff admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:15.690723 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c80d8a-432b-4ab0-9ba5-c515d16b707f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:15.691859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c80d8a-432b-4ab0-9ba5-c515d16b707f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:15.692427 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c80d8a-432b-4ab0-9ba5-c515d16b707f admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:15.692724 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c80d8a-432b-4ab0-9ba5-c515d16b707f admin admin] PATH_INFO: `/v3/projects/525245253a2d49c48ef009a4d6c3cd51` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:15.694045 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c80d8a-432b-4ab0-9ba5-c515d16b707f admin admin] RBAC: Authorizing `identity:delete_project(project_id=525245253a2d49c48ef009a4d6c3cd51)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:15.695987 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c80d8a-432b-4ab0-9ba5-c515d16b707f admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:15.725644 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1351/2672] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:00:15 2022] POST /identity/v3/users => generated 297 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:00:15.729637 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f992b2-9e91-4fa6-9649-3ec92886f4a2 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:15.739618 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f992b2-9e91-4fa6-9649-3ec92886f4a2 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:15.744198 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6efc4-bc86-4950-bbb0-2a4e419acc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:15.744837 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6efc4-bc86-4950-bbb0-2a4e419acc84 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:15.745118 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6efc4-bc86-4950-bbb0-2a4e419acc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:15.745398 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6efc4-bc86-4950-bbb0-2a4e419acc84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:15.746028 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a6efc4-bc86-4950-bbb0-2a4e419acc84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:15.748248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a6efc4-bc86-4950-bbb0-2a4e419acc84 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:15.760723 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1352/2673] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:00:15.763578 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6efc4-bc86-4950-bbb0-2a4e419acc84 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:15.772934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6efc4-bc86-4950-bbb0-2a4e419acc84 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:15.777561 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cda05-57c2-4015-b6df-0f3da8f4a612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:15.778215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cda05-57c2-4015-b6df-0f3da8f4a612 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:15.778501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cda05-57c2-4015-b6df-0f3da8f4a612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:15.778782 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cda05-57c2-4015-b6df-0f3da8f4a612 admin admin] PATH_INFO: `/v3/projects/a178795229e84f4f8b550e6013700fe7/users/37a9848ad4394f8dbfe846fa76665e51/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:15.794997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8cda05-57c2-4015-b6df-0f3da8f4a612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a178795229e84f4f8b550e6013700fe7, user_id=37a9848ad4394f8dbfe846fa76665e51, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:15.797749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8cda05-57c2-4015-b6df-0f3da8f4a612 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:15.908566 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1353/2674] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:00:15 2022] PUT /identity/v3/projects/a178795229e84f4f8b550e6013700fe7/users/37a9848ad4394f8dbfe846fa76665e51/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:00:15.912198 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cda05-57c2-4015-b6df-0f3da8f4a612 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:15.920582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-24c80d8a-432b-4ab0-9ba5-c515d16b707f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '525245253a2d49c48ef009a4d6c3cd51', 'request_id': 'req-24c80d8a-432b-4ab0-9ba5-c515d16b707f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:15.927083 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-24c80d8a-432b-4ab0-9ba5-c515d16b707f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '525245253a2d49c48ef009a4d6c3cd51', 'request_id': 'req-24c80d8a-432b-4ab0-9ba5-c515d16b707f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:15.932939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-24c80d8a-432b-4ab0-9ba5-c515d16b707f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '525245253a2d49c48ef009a4d6c3cd51', 'request_id': 'req-24c80d8a-432b-4ab0-9ba5-c515d16b707f'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:15.982805 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cda05-57c2-4015-b6df-0f3da8f4a612 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:16.011956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-24c80d8a-432b-4ab0-9ba5-c515d16b707f admin admin] The token cache is being invalidate because project 525245253a2d49c48ef009a4d6c3cd51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:00:16.012258 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-24c80d8a-432b-4ab0-9ba5-c515d16b707f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:16.013371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-24c80d8a-432b-4ab0-9ba5-c515d16b707f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:16.013630 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:16.014976 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1322/2675] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:00:15 2022] DELETE /identity/v3/projects/525245253a2d49c48ef009a4d6c3cd51 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:00:16.021928 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dc7be-81c0-4cba-8720-d0ded347c2fd None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:16.022221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dc7be-81c0-4cba-8720-d0ded347c2fd None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:16.022530 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dc7be-81c0-4cba-8720-d0ded347c2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:16.044085 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c184a2-4530-4d56-a649-3565e0623ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:16.044357 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-ec8dc7be-81c0-4cba-8720-d0ded347c2fd None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:16.044748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c184a2-4530-4d56-a649-3565e0623ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:16.045031 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c184a2-4530-4d56-a649-3565e0623ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:16.045308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c184a2-4530-4d56-a649-3565e0623ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:16.045452 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ec8dc7be-81c0-4cba-8720-d0ded347c2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:16.045959 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c184a2-4530-4d56-a649-3565e0623ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:16.047632 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c184a2-4530-4d56-a649-3565e0623ef9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:16.053858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1323/2676] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:16.057754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1354/2677] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:00:16.062036 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99c788-5a6e-473c-829f-e9cda4cf18e5 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:16.062197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c184a2-4530-4d56-a649-3565e0623ef9 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:16.062378 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99c788-5a6e-473c-829f-e9cda4cf18e5 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:16.062656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99c788-5a6e-473c-829f-e9cda4cf18e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:16.063903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a9c184a2-4530-4d56-a649-3565e0623ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:16.076468 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c184a2-4530-4d56-a649-3565e0623ef9 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:16.081142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab337baa-0c0e-4f47-bd0b-b1248ee50e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:16.087788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab337baa-0c0e-4f47-bd0b-b1248ee50e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:16.087788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab337baa-0c0e-4f47-bd0b-b1248ee50e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:16.087788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab337baa-0c0e-4f47-bd0b-b1248ee50e16 admin admin] PATH_INFO: `/v3/projects/a178795229e84f4f8b550e6013700fe7/users/37a9848ad4394f8dbfe846fa76665e51/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:16.088257 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-3c99c788-5a6e-473c-829f-e9cda4cf18e5 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:16.088496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab337baa-0c0e-4f47-bd0b-b1248ee50e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a178795229e84f4f8b550e6013700fe7, user_id=37a9848ad4394f8dbfe846fa76665e51, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:16.088600 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3c99c788-5a6e-473c-829f-e9cda4cf18e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:16.088695 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab337baa-0c0e-4f47-bd0b-b1248ee50e16 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:16.094442 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1324/2678] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:16.097801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1355/2679] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:00:16 2022] PUT /identity/v3/projects/a178795229e84f4f8b550e6013700fe7/users/37a9848ad4394f8dbfe846fa76665e51/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:00:16.098409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99c788-5a6e-473c-829f-e9cda4cf18e5 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:16.107529 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99c788-5a6e-473c-829f-e9cda4cf18e5 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:16.107889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b82a3-a61e-4d7f-94ce-ebe7ce45ec87 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:16.108525 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b82a3-a61e-4d7f-94ce-ebe7ce45ec87 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:16.109002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b82a3-a61e-4d7f-94ce-ebe7ce45ec87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:16.137465 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-787b82a3-a61e-4d7f-94ce-ebe7ce45ec87 None None] MFA Rules not processed for user `37a9848ad4394f8dbfe846fa76665e51`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:16.140001 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-787b82a3-a61e-4d7f-94ce-ebe7ce45ec87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:16.166319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc2a98-844d-43ea-9942-e0597e29a7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:16.166931 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc2a98-844d-43ea-9942-e0597e29a7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:16.167212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc2a98-844d-43ea-9942-e0597e29a7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:16.167485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc2a98-844d-43ea-9942-e0597e29a7c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:16.168128 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc2a98-844d-43ea-9942-e0597e29a7c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:00:16.168482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc2a98-844d-43ea-9942-e0597e29a7c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:16.170124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc2a98-844d-43ea-9942-e0597e29a7c4 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:16.181654 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1325/2680] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 17:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:00:16.186465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc2a98-844d-43ea-9942-e0597e29a7c4 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:16.195835 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc2a98-844d-43ea-9942-e0597e29a7c4 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:16.200971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541e05c-773d-4897-b657-f0e2ab2311a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:16.201547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541e05c-773d-4897-b657-f0e2ab2311a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:16.201825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541e05c-773d-4897-b657-f0e2ab2311a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:16.202093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541e05c-773d-4897-b657-f0e2ab2311a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:16.202750 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541e05c-773d-4897-b657-f0e2ab2311a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:16.204637 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541e05c-773d-4897-b657-f0e2ab2311a0 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:16.207553 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1356/2681] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:16 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:16.213142 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:16.219288 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b82a3-a61e-4d7f-94ce-ebe7ce45ec87 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:16.220685 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-787b82a3-a61e-4d7f-94ce-ebe7ce45ec87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:16.277558 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b82a3-a61e-4d7f-94ce-ebe7ce45ec87 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:16.281148 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c34c1d-f3a7-4e5f-afa5-a5816253cba8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:16.281756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c34c1d-f3a7-4e5f-afa5-a5816253cba8 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:16.282036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c34c1d-f3a7-4e5f-afa5-a5816253cba8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:16.282317 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c34c1d-f3a7-4e5f-afa5-a5816253cba8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:16.291180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c34c1d-f3a7-4e5f-afa5-a5816253cba8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:16.292624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c34c1d-f3a7-4e5f-afa5-a5816253cba8 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:16.302123 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1357/2682] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:16 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:00:16.327161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1326/2683] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:00:16 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:00:16.331922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c34c1d-f3a7-4e5f-afa5-a5816253cba8 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:16.342429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c34c1d-f3a7-4e5f-afa5-a5816253cba8 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:16.382543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:16.398729 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f5f33-583e-423b-be6c-53253ebd21b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:16.399342 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f5f33-583e-423b-be6c-53253ebd21b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:16.399624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f5f33-583e-423b-be6c-53253ebd21b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:16.399900 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f5f33-583e-423b-be6c-53253ebd21b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:16.400564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f5f33-583e-423b-be6c-53253ebd21b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:16.402478 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f5f33-583e-423b-be6c-53253ebd21b1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:16.453680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541e05c-773d-4897-b657-f0e2ab2311a0 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:16.509802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1358/2684] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:00:16 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:00:16.510403 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541e05c-773d-4897-b657-f0e2ab2311a0 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:16.513913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f5f33-583e-423b-be6c-53253ebd21b1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:16.514458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b067aa-0c00-419a-984b-07fca5d0795e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:16.515061 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b067aa-0c00-419a-984b-07fca5d0795e service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:16.515361 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b067aa-0c00-419a-984b-07fca5d0795e service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:16.515619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b067aa-0c00-419a-984b-07fca5d0795e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:16.517364 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-81b067aa-0c00-419a-984b-07fca5d0795e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:16.524039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f5f33-583e-423b-be6c-53253ebd21b1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:16.528889 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb75322-098b-494a-aa4e-2cc9cb018262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:16.529484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb75322-098b-494a-aa4e-2cc9cb018262 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:16.529711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb75322-098b-494a-aa4e-2cc9cb018262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:16.529987 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb75322-098b-494a-aa4e-2cc9cb018262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:16.530628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb75322-098b-494a-aa4e-2cc9cb018262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:16.532279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb75322-098b-494a-aa4e-2cc9cb018262 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:16.542365 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1359/2685] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:00:16.546857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb75322-098b-494a-aa4e-2cc9cb018262 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:16.556486 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb75322-098b-494a-aa4e-2cc9cb018262 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:16.562215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea656b-3191-4351-ad8c-e4dda4be3aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:16.562749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea656b-3191-4351-ad8c-e4dda4be3aae admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:16.563292 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea656b-3191-4351-ad8c-e4dda4be3aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:16.563292 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea656b-3191-4351-ad8c-e4dda4be3aae admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:16.563954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea656b-3191-4351-ad8c-e4dda4be3aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:16.565687 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea656b-3191-4351-ad8c-e4dda4be3aae admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:16.576251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1360/2686] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:00:16.579459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b067aa-0c00-419a-984b-07fca5d0795e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:16.580322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea656b-3191-4351-ad8c-e4dda4be3aae admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:16.581238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b067aa-0c00-419a-984b-07fca5d0795e service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:16.590028 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea656b-3191-4351-ad8c-e4dda4be3aae admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:16.590403 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1327/2687] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:00:16.597360 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a37679-fdde-4f35-9e98-03bda939e467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:16.598126 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a37679-fdde-4f35-9e98-03bda939e467 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:16.598294 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a37679-fdde-4f35-9e98-03bda939e467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:16.598570 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a37679-fdde-4f35-9e98-03bda939e467 admin admin] PATH_INFO: `/v3/projects/19ac17f7efa644819e75576a06bb2499/users/9d1076aed12f4d22a34df10ed0d8e462/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:16.614006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a37679-fdde-4f35-9e98-03bda939e467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ac17f7efa644819e75576a06bb2499, user_id=9d1076aed12f4d22a34df10ed0d8e462, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:16.616729 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a37679-fdde-4f35-9e98-03bda939e467 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:16.685273 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1361/2688] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:00:16 2022] PUT /identity/v3/projects/19ac17f7efa644819e75576a06bb2499/users/9d1076aed12f4d22a34df10ed0d8e462/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:00:16.690410 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e54ba-5570-49af-83c6-a51bba159cf9 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:16.690695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e54ba-5570-49af-83c6-a51bba159cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:16.690962 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e54ba-5570-49af-83c6-a51bba159cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:16.710569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-6f2e54ba-5570-49af-83c6-a51bba159cf9 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:16.711758 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6f2e54ba-5570-49af-83c6-a51bba159cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:16.776182 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1328/2689] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:00:16.787036 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a37679-fdde-4f35-9e98-03bda939e467 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:16.788842 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-09a37679-fdde-4f35-9e98-03bda939e467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:16.847894 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a37679-fdde-4f35-9e98-03bda939e467 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:16.849658 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a8ec4-c884-461a-ab85-c532974cfb7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:16.850258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a8ec4-c884-461a-ab85-c532974cfb7d service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:16.850538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a8ec4-c884-461a-ab85-c532974cfb7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:16.850812 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a8ec4-c884-461a-ab85-c532974cfb7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:16.860306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a8ec4-c884-461a-ab85-c532974cfb7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:16.861814 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a8ec4-c884-461a-ab85-c532974cfb7d service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:16.871671 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1362/2690] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:00:17.110465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e54ba-5570-49af-83c6-a51bba159cf9 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:17.120189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e54ba-5570-49af-83c6-a51bba159cf9 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:17.125430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92284c25-81ab-453e-ad1d-1ce340ac9d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:17.131551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92284c25-81ab-453e-ad1d-1ce340ac9d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:17.131551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92284c25-81ab-453e-ad1d-1ce340ac9d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:17.131551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92284c25-81ab-453e-ad1d-1ce340ac9d58 admin admin] PATH_INFO: `/v3/users/d42f2da0912941bfb68a2691eac7b8c7` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:17.131551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92284c25-81ab-453e-ad1d-1ce340ac9d58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d42f2da0912941bfb68a2691eac7b8c7)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:17.131551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92284c25-81ab-453e-ad1d-1ce340ac9d58 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:17.359563 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:17.367615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-92284c25-81ab-453e-ad1d-1ce340ac9d58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd42f2da0912941bfb68a2691eac7b8c7'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:17.368280 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-92284c25-81ab-453e-ad1d-1ce340ac9d58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd42f2da0912941bfb68a2691eac7b8c7'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:17.368905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-92284c25-81ab-453e-ad1d-1ce340ac9d58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd42f2da0912941bfb68a2691eac7b8c7'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:17.376337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-92284c25-81ab-453e-ad1d-1ce340ac9d58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd42f2da0912941bfb68a2691eac7b8c7'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:17.451225 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-92284c25-81ab-453e-ad1d-1ce340ac9d58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd42f2da0912941bfb68a2691eac7b8c7'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:17.474605 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1329/2691] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:00:17 2022] DELETE /identity/v3/users/d42f2da0912941bfb68a2691eac7b8c7 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:17.479183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0eb55-dee7-41ad-80a2-55cf814b66eb None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:17.484056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0eb55-dee7-41ad-80a2-55cf814b66eb None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:17.484056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0eb55-dee7-41ad-80a2-55cf814b66eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:17.499059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-83a0eb55-dee7-41ad-80a2-55cf814b66eb None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:17.500253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-83a0eb55-dee7-41ad-80a2-55cf814b66eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:17.560665 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1363/2692] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:17.570711 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92284c25-81ab-453e-ad1d-1ce340ac9d58 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:17.572379 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-92284c25-81ab-453e-ad1d-1ce340ac9d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:17.633938 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92284c25-81ab-453e-ad1d-1ce340ac9d58 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:17.637665 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ab69c-3000-460d-b93d-a2d92a8e0041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:17.638263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ab69c-3000-460d-b93d-a2d92a8e0041 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:17.638540 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ab69c-3000-460d-b93d-a2d92a8e0041 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:17.638809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ab69c-3000-460d-b93d-a2d92a8e0041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:17.648220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ab69c-3000-460d-b93d-a2d92a8e0041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:17.650470 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ab69c-3000-460d-b93d-a2d92a8e0041 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:17.659846 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1330/2693] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:00:18.092787 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0eb55-dee7-41ad-80a2-55cf814b66eb None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:18.094476 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-83a0eb55-dee7-41ad-80a2-55cf814b66eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:18.108865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0eb55-dee7-41ad-80a2-55cf814b66eb None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:18.113742 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:18.114365 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:18.114654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:18.114931 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2 admin admin] PATH_INFO: `/v3/projects/865a94937481403ebef955ea3dd821cd` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:18.116813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=865a94937481403ebef955ea3dd821cd)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:18.119544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:18.148592 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:18.235110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '865a94937481403ebef955ea3dd821cd', 'request_id': 'req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:18.243022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '865a94937481403ebef955ea3dd821cd', 'request_id': 'req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:18.249610 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '865a94937481403ebef955ea3dd821cd', 'request_id': 'req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:18.320496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2 admin admin] The token cache is being invalidate because project 865a94937481403ebef955ea3dd821cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:00:18.320496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:18.320974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:18.322742 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1364/2694] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:00:18 2022] DELETE /identity/v3/projects/865a94937481403ebef955ea3dd821cd => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:18.326996 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ab69c-3000-460d-b93d-a2d92a8e0041 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:18.337284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ab69c-3000-460d-b93d-a2d92a8e0041 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:18.398244 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f657be1-8672-4c21-901f-4f6674896cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:18.398934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f657be1-8672-4c21-901f-4f6674896cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:18.399144 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f657be1-8672-4c21-901f-4f6674896cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:18.399418 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f657be1-8672-4c21-901f-4f6674896cbe admin admin] PATH_INFO: `/v3/users/37a9848ad4394f8dbfe846fa76665e51` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:18.400686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f657be1-8672-4c21-901f-4f6674896cbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=37a9848ad4394f8dbfe846fa76665e51)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:18.402510 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f657be1-8672-4c21-901f-4f6674896cbe admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:18.585048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9f657be1-8672-4c21-901f-4f6674896cbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37a9848ad4394f8dbfe846fa76665e51'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:18.585743 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9f657be1-8672-4c21-901f-4f6674896cbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37a9848ad4394f8dbfe846fa76665e51'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:18.586730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9f657be1-8672-4c21-901f-4f6674896cbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37a9848ad4394f8dbfe846fa76665e51'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:18.594132 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9f657be1-8672-4c21-901f-4f6674896cbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37a9848ad4394f8dbfe846fa76665e51'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:18.596670 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:18.691869 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9f657be1-8672-4c21-901f-4f6674896cbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37a9848ad4394f8dbfe846fa76665e51'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:18.712363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1331/2695] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:00:18 2022] DELETE /identity/v3/users/37a9848ad4394f8dbfe846fa76665e51 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:19.276158 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:19.277874 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:19.300016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a37ec-1a51-48b8-8bcc-439dbf9c44d2 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:19.367389 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc143ec-fe81-40d8-92f3-e4a6b5a79c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:19.368060 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc143ec-fe81-40d8-92f3-e4a6b5a79c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:19.368368 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc143ec-fe81-40d8-92f3-e4a6b5a79c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:19.368827 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc143ec-fe81-40d8-92f3-e4a6b5a79c4e admin admin] PATH_INFO: `/v3/projects/a178795229e84f4f8b550e6013700fe7` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:19.369960 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc143ec-fe81-40d8-92f3-e4a6b5a79c4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a178795229e84f4f8b550e6013700fe7)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:19.372375 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc143ec-fe81-40d8-92f3-e4a6b5a79c4e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:19.487870 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4bc143ec-fe81-40d8-92f3-e4a6b5a79c4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a178795229e84f4f8b550e6013700fe7', 'request_id': 'req-4bc143ec-fe81-40d8-92f3-e4a6b5a79c4e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:19.494041 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4bc143ec-fe81-40d8-92f3-e4a6b5a79c4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a178795229e84f4f8b550e6013700fe7', 'request_id': 'req-4bc143ec-fe81-40d8-92f3-e4a6b5a79c4e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:19.501947 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4bc143ec-fe81-40d8-92f3-e4a6b5a79c4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a178795229e84f4f8b550e6013700fe7', 'request_id': 'req-4bc143ec-fe81-40d8-92f3-e4a6b5a79c4e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:19.606343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4bc143ec-fe81-40d8-92f3-e4a6b5a79c4e admin admin] The token cache is being invalidate because project a178795229e84f4f8b550e6013700fe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:00:19.609433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4bc143ec-fe81-40d8-92f3-e4a6b5a79c4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:19.609433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4bc143ec-fe81-40d8-92f3-e4a6b5a79c4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:19.609641 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1365/2696] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:00:19 2022] DELETE /identity/v3/projects/a178795229e84f4f8b550e6013700fe7 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:19.616240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e88da-8997-49ee-9352-003b0ca29ef9 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:19.616714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e88da-8997-49ee-9352-003b0ca29ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:19.617147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e88da-8997-49ee-9352-003b0ca29ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:19.647538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-741e88da-8997-49ee-9352-003b0ca29ef9 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:19.651168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-741e88da-8997-49ee-9352-003b0ca29ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:19.777175 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1332/2697] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:19.785838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce2bcf-c1ee-4a07-a481-8307df12282e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:19.786148 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce2bcf-c1ee-4a07-a481-8307df12282e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:19.786381 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce2bcf-c1ee-4a07-a481-8307df12282e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:19.811002 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b8ce2bcf-c1ee-4a07-a481-8307df12282e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:19.812305 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b8ce2bcf-c1ee-4a07-a481-8307df12282e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:19.825825 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1366/2698] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:00:19.830328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e88da-8997-49ee-9352-003b0ca29ef9 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:19.841691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e88da-8997-49ee-9352-003b0ca29ef9 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:19.847750 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a48c5-12db-4af6-8e69-32ad35358834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:19.848554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a48c5-12db-4af6-8e69-32ad35358834 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:19.849388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a48c5-12db-4af6-8e69-32ad35358834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:19.849779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a48c5-12db-4af6-8e69-32ad35358834 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:19.850550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a48c5-12db-4af6-8e69-32ad35358834 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:00:19.850955 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a48c5-12db-4af6-8e69-32ad35358834 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:19.852880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a48c5-12db-4af6-8e69-32ad35358834 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:19.868866 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1333/2699] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 17:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:00:19.873584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce2bcf-c1ee-4a07-a481-8307df12282e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:19.884383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce2bcf-c1ee-4a07-a481-8307df12282e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:19.890668 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a8f32-11ab-4d3c-9437-0f789ee145c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:19.891287 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a8f32-11ab-4d3c-9437-0f789ee145c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:19.891567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a8f32-11ab-4d3c-9437-0f789ee145c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:19.891920 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a8f32-11ab-4d3c-9437-0f789ee145c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:19.892519 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a8f32-11ab-4d3c-9437-0f789ee145c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:19.894785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a8f32-11ab-4d3c-9437-0f789ee145c4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:20.011417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1367/2700] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:00:19 2022] POST /identity/v3/projects => generated 361 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:00:20.013084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a48c5-12db-4af6-8e69-32ad35358834 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:20.025845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a48c5-12db-4af6-8e69-32ad35358834 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:20.096884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994e092-5e1a-43a3-b11b-d100ee9e92e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:20.097507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994e092-5e1a-43a3-b11b-d100ee9e92e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:20.097791 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994e092-5e1a-43a3-b11b-d100ee9e92e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:20.098067 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994e092-5e1a-43a3-b11b-d100ee9e92e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:20.099043 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994e092-5e1a-43a3-b11b-d100ee9e92e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:20.100809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994e092-5e1a-43a3-b11b-d100ee9e92e6 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:20.196301 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1334/2701] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:00:20 2022] POST /identity/v3/users => generated 281 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:00:20.200073 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a8f32-11ab-4d3c-9437-0f789ee145c4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:20.211918 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a8f32-11ab-4d3c-9437-0f789ee145c4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:20.218809 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a40cf5-b0e1-4b16-b4c0-f57ac5da1b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:20.219445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a40cf5-b0e1-4b16-b4c0-f57ac5da1b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:20.222320 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a40cf5-b0e1-4b16-b4c0-f57ac5da1b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:20.222320 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a40cf5-b0e1-4b16-b4c0-f57ac5da1b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:20.222320 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a40cf5-b0e1-4b16-b4c0-f57ac5da1b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:20.222581 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a40cf5-b0e1-4b16-b4c0-f57ac5da1b77 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:20.234688 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1368/2702] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:00:20.239032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994e092-5e1a-43a3-b11b-d100ee9e92e6 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:20.253106 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994e092-5e1a-43a3-b11b-d100ee9e92e6 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:20.260282 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6fc8f-2dca-4bdb-98b4-fbb7c89d0467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:20.261569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6fc8f-2dca-4bdb-98b4-fbb7c89d0467 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:20.261954 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6fc8f-2dca-4bdb-98b4-fbb7c89d0467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:20.263156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6fc8f-2dca-4bdb-98b4-fbb7c89d0467 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:20.263618 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6fc8f-2dca-4bdb-98b4-fbb7c89d0467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:20.265441 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6fc8f-2dca-4bdb-98b4-fbb7c89d0467 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:20.279752 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1335/2703] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:00:20.284433 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a40cf5-b0e1-4b16-b4c0-f57ac5da1b77 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:20.290167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:20.297692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a40cf5-b0e1-4b16-b4c0-f57ac5da1b77 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:20.303940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870f08a-a20f-4c3f-8853-6983e6b512ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:20.303940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870f08a-a20f-4c3f-8853-6983e6b512ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:20.303940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870f08a-a20f-4c3f-8853-6983e6b512ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:20.303940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870f08a-a20f-4c3f-8853-6983e6b512ea admin admin] PATH_INFO: `/v3/projects/0ea4325e162d49fe9983faf1b313712e/users/e827b956ad5f40578a79b9d09d6eecd4/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:20.321793 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870f08a-a20f-4c3f-8853-6983e6b512ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea4325e162d49fe9983faf1b313712e, user_id=e827b956ad5f40578a79b9d09d6eecd4, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:20.322823 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870f08a-a20f-4c3f-8853-6983e6b512ea admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:20.407486 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1369/2704] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:00:20 2022] PUT /identity/v3/projects/0ea4325e162d49fe9983faf1b313712e/users/e827b956ad5f40578a79b9d09d6eecd4/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:20.418010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dab85-83d5-4c56-a8b7-b7e60ff72e78 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:20.418338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dab85-83d5-4c56-a8b7-b7e60ff72e78 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:20.418528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dab85-83d5-4c56-a8b7-b7e60ff72e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:20.453678 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-d70dab85-83d5-4c56-a8b7-b7e60ff72e78 None None] MFA Rules not processed for user `e827b956ad5f40578a79b9d09d6eecd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:20.454963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d70dab85-83d5-4c56-a8b7-b7e60ff72e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:20.498661 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:20.548152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1336/2705] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:00:20.593778 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870f08a-a20f-4c3f-8853-6983e6b512ea admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:20.595871 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0870f08a-a20f-4c3f-8853-6983e6b512ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:20.664006 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870f08a-a20f-4c3f-8853-6983e6b512ea admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:20.667930 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da7590-3c0c-4981-b230-4ab3caca2e2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:20.668849 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da7590-3c0c-4981-b230-4ab3caca2e2b service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:20.669235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da7590-3c0c-4981-b230-4ab3caca2e2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:20.669633 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da7590-3c0c-4981-b230-4ab3caca2e2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:20.679963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da7590-3c0c-4981-b230-4ab3caca2e2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:20.681749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da7590-3c0c-4981-b230-4ab3caca2e2b service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:20.692540 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1370/2706] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:00:20.708388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70dab85-83d5-4c56-a8b7-b7e60ff72e78 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:20.784089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70dab85-83d5-4c56-a8b7-b7e60ff72e78 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:20.789392 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6a594-b2f0-4d17-afb6-178ec5a4eeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:20.790036 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6a594-b2f0-4d17-afb6-178ec5a4eeaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:20.790313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6a594-b2f0-4d17-afb6-178ec5a4eeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:20.790615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6a594-b2f0-4d17-afb6-178ec5a4eeaa admin admin] PATH_INFO: `/v3/users/e827b956ad5f40578a79b9d09d6eecd4` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:20.791910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6a594-b2f0-4d17-afb6-178ec5a4eeaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e827b956ad5f40578a79b9d09d6eecd4)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:20.795308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6a594-b2f0-4d17-afb6-178ec5a4eeaa admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:20.963299 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-66c6a594-b2f0-4d17-afb6-178ec5a4eeaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e827b956ad5f40578a79b9d09d6eecd4'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:20.964023 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-66c6a594-b2f0-4d17-afb6-178ec5a4eeaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e827b956ad5f40578a79b9d09d6eecd4'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:20.965338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-66c6a594-b2f0-4d17-afb6-178ec5a4eeaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e827b956ad5f40578a79b9d09d6eecd4'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:20.973339 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-66c6a594-b2f0-4d17-afb6-178ec5a4eeaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e827b956ad5f40578a79b9d09d6eecd4'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:21.056635 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-66c6a594-b2f0-4d17-afb6-178ec5a4eeaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e827b956ad5f40578a79b9d09d6eecd4'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:21.077208 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1337/2707] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:00:20 2022] DELETE /identity/v3/users/e827b956ad5f40578a79b9d09d6eecd4 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:00:21.082200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d890322-8515-4503-948f-029c827d716b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:21.084246 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d890322-8515-4503-948f-029c827d716b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:21.084544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d890322-8515-4503-948f-029c827d716b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:21.103689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8d890322-8515-4503-948f-029c827d716b None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:21.104952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8d890322-8515-4503-948f-029c827d716b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:21.175832 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1371/2708] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:21.187356 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6a594-b2f0-4d17-afb6-178ec5a4eeaa admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:21.199544 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-66c6a594-b2f0-4d17-afb6-178ec5a4eeaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:21.246527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6a594-b2f0-4d17-afb6-178ec5a4eeaa admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:21.255564 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbddf333-066e-4319-a6bb-937987b8404a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:21.257313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddf333-066e-4319-a6bb-937987b8404a service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:21.257313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddf333-066e-4319-a6bb-937987b8404a service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:21.257543 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddf333-066e-4319-a6bb-937987b8404a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:21.264338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:21.268387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddf333-066e-4319-a6bb-937987b8404a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:21.269874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddf333-066e-4319-a6bb-937987b8404a service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:21.281939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1338/2709] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:00:21.904942 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d890322-8515-4503-948f-029c827d716b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:21.908953 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8d890322-8515-4503-948f-029c827d716b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:21.918275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d890322-8515-4503-948f-029c827d716b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:21.922876 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964cd20-d860-405e-875f-5f185f3f17da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:21.923509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964cd20-d860-405e-875f-5f185f3f17da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:21.923796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964cd20-d860-405e-875f-5f185f3f17da admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:21.924065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964cd20-d860-405e-875f-5f185f3f17da admin admin] PATH_INFO: `/v3/projects/0ea4325e162d49fe9983faf1b313712e` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:21.925497 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964cd20-d860-405e-875f-5f185f3f17da admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea4325e162d49fe9983faf1b313712e)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:21.927409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964cd20-d860-405e-875f-5f185f3f17da admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:22.049952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c964cd20-d860-405e-875f-5f185f3f17da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea4325e162d49fe9983faf1b313712e', 'request_id': 'req-c964cd20-d860-405e-875f-5f185f3f17da'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:22.057256 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c964cd20-d860-405e-875f-5f185f3f17da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea4325e162d49fe9983faf1b313712e', 'request_id': 'req-c964cd20-d860-405e-875f-5f185f3f17da'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:22.065014 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c964cd20-d860-405e-875f-5f185f3f17da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea4325e162d49fe9983faf1b313712e', 'request_id': 'req-c964cd20-d860-405e-875f-5f185f3f17da'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:22.162487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c964cd20-d860-405e-875f-5f185f3f17da admin admin] The token cache is being invalidate because project 0ea4325e162d49fe9983faf1b313712e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:00:22.163155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c964cd20-d860-405e-875f-5f185f3f17da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:22.164152 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c964cd20-d860-405e-875f-5f185f3f17da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:22.166528 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1372/2710] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:00:21 2022] DELETE /identity/v3/projects/0ea4325e162d49fe9983faf1b313712e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:22.172786 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a4957-0d29-48cc-935f-03d49d10a220 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:22.173701 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a4957-0d29-48cc-935f-03d49d10a220 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:22.174182 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a4957-0d29-48cc-935f-03d49d10a220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:22.204457 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-392a4957-0d29-48cc-935f-03d49d10a220 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:22.205882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-392a4957-0d29-48cc-935f-03d49d10a220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:22.339525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1339/2711] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:00:22.344104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca65c8-5168-48dc-877e-22928d182f90 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:22.344686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca65c8-5168-48dc-877e-22928d182f90 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:22.345051 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca65c8-5168-48dc-877e-22928d182f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:22.356866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:22.375801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8fca65c8-5168-48dc-877e-22928d182f90 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:22.376993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8fca65c8-5168-48dc-877e-22928d182f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:22.388378 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1373/2712] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:22.393097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a4957-0d29-48cc-935f-03d49d10a220 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:22.405594 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a4957-0d29-48cc-935f-03d49d10a220 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:22.415844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4403b3-2151-4263-b5c7-4bf92aad1ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:22.417216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4403b3-2151-4263-b5c7-4bf92aad1ddf admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:22.417216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4403b3-2151-4263-b5c7-4bf92aad1ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:22.417435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4403b3-2151-4263-b5c7-4bf92aad1ddf admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:22.418867 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4403b3-2151-4263-b5c7-4bf92aad1ddf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:00:22.419616 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4403b3-2151-4263-b5c7-4bf92aad1ddf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:22.421464 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4403b3-2151-4263-b5c7-4bf92aad1ddf admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:22.437456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1340/2713] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 17:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:00:22.441621 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca65c8-5168-48dc-877e-22928d182f90 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:22.459358 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca65c8-5168-48dc-877e-22928d182f90 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:22.466422 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1046c9b-1e0c-4f41-a58c-3179b659da97 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:22.466721 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1046c9b-1e0c-4f41-a58c-3179b659da97 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:22.466964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1046c9b-1e0c-4f41-a58c-3179b659da97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:22.471485 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1acd6fe-f69d-47fd-9ef8-b3cb591db92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:22.472238 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acd6fe-f69d-47fd-9ef8-b3cb591db92d admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:22.472581 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acd6fe-f69d-47fd-9ef8-b3cb591db92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:22.472853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acd6fe-f69d-47fd-9ef8-b3cb591db92d admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:22.473502 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1acd6fe-f69d-47fd-9ef8-b3cb591db92d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:22.475590 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1acd6fe-f69d-47fd-9ef8-b3cb591db92d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:22.504946 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-c1046c9b-1e0c-4f41-a58c-3179b659da97 None None] MFA Rules not processed for user `9d1076aed12f4d22a34df10ed0d8e462`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:22.506771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c1046c9b-1e0c-4f41-a58c-3179b659da97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:22.576950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1341/2714] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:22.591119 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1046c9b-1e0c-4f41-a58c-3179b659da97 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:22.592630 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c1046c9b-1e0c-4f41-a58c-3179b659da97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:22.601852 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1374/2715] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:00:22 2022] POST /identity/v3/projects => generated 391 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:00:22.607552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1acd6fe-f69d-47fd-9ef8-b3cb591db92d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:22.681170 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1acd6fe-f69d-47fd-9ef8-b3cb591db92d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:22.682582 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1046c9b-1e0c-4f41-a58c-3179b659da97 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:22.686916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8707ef-e8d6-496f-801a-ace6ed837b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:22.687689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8707ef-e8d6-496f-801a-ace6ed837b58 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:22.687977 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8707ef-e8d6-496f-801a-ace6ed837b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:22.688253 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8707ef-e8d6-496f-801a-ace6ed837b58 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:22.688930 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8707ef-e8d6-496f-801a-ace6ed837b58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:22.689798 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:22.690850 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8707ef-e8d6-496f-801a-ace6ed837b58 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:22.757624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf8f02-4264-436e-bb1c-23693f551a63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:22.758241 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf8f02-4264-436e-bb1c-23693f551a63 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:22.759168 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf8f02-4264-436e-bb1c-23693f551a63 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:22.759644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf8f02-4264-436e-bb1c-23693f551a63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:22.812286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1375/2716] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:00:22 2022] POST /identity/v3/users => generated 296 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:00:22.817891 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8707ef-e8d6-496f-801a-ace6ed837b58 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:22.829388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8707ef-e8d6-496f-801a-ace6ed837b58 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:22.829697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf8f02-4264-436e-bb1c-23693f551a63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:22.831440 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf8f02-4264-436e-bb1c-23693f551a63 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:22.834600 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f2301-4853-4da4-98e3-6bda3b3ac50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:22.835402 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f2301-4853-4da4-98e3-6bda3b3ac50e admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:22.835685 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f2301-4853-4da4-98e3-6bda3b3ac50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:22.836011 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f2301-4853-4da4-98e3-6bda3b3ac50e admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:22.836802 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f2301-4853-4da4-98e3-6bda3b3ac50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:22.842344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f2301-4853-4da4-98e3-6bda3b3ac50e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:22.850373 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1376/2717] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:00:22.853072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1342/2718] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:00:22.854256 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f2301-4853-4da4-98e3-6bda3b3ac50e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:22.866118 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f2301-4853-4da4-98e3-6bda3b3ac50e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:22.871021 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360027c-7917-4bc6-a620-68c7f7531cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:22.874355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360027c-7917-4bc6-a620-68c7f7531cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:22.874355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360027c-7917-4bc6-a620-68c7f7531cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:22.874355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360027c-7917-4bc6-a620-68c7f7531cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:22.874355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360027c-7917-4bc6-a620-68c7f7531cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:22.875321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360027c-7917-4bc6-a620-68c7f7531cd7 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:22.889568 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1377/2719] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:00:22.893897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf8f02-4264-436e-bb1c-23693f551a63 service nova] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:22.905737 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf8f02-4264-436e-bb1c-23693f551a63 service nova] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:22.915362 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba9e58-39e8-4a6c-a587-3b8dee362fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:22.916050 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba9e58-39e8-4a6c-a587-3b8dee362fa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:22.916331 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba9e58-39e8-4a6c-a587-3b8dee362fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:22.916774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba9e58-39e8-4a6c-a587-3b8dee362fa2 admin admin] PATH_INFO: `/v3/projects/a7a25e763a1b4d3b9ca1cb0deb04e365/users/d7389e2ff6f042a58e6005412ff69c19/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:22.933697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba9e58-39e8-4a6c-a587-3b8dee362fa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a25e763a1b4d3b9ca1cb0deb04e365, user_id=d7389e2ff6f042a58e6005412ff69c19, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:22.936958 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba9e58-39e8-4a6c-a587-3b8dee362fa2 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:23.013809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1343/2720] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:00:22 2022] PUT /identity/v3/projects/a7a25e763a1b4d3b9ca1cb0deb04e365/users/d7389e2ff6f042a58e6005412ff69c19/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:00:23.081069 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789f708-cbc8-492a-a7ab-1ad837a57cd3 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:23.081572 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789f708-cbc8-492a-a7ab-1ad837a57cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:23.081686 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789f708-cbc8-492a-a7ab-1ad837a57cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:23.110697 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7789f708-cbc8-492a-a7ab-1ad837a57cd3 None None] MFA Rules not processed for user `d7389e2ff6f042a58e6005412ff69c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:23.112016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7789f708-cbc8-492a-a7ab-1ad837a57cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:23.185530 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1378/2721] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:23.195255 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba9e58-39e8-4a6c-a587-3b8dee362fa2 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:23.197082 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-e1ba9e58-39e8-4a6c-a587-3b8dee362fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:23.270315 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba9e58-39e8-4a6c-a587-3b8dee362fa2 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:23.274177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf50233-4507-4117-aaf2-9109acf1dc84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:23.274784 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf50233-4507-4117-aaf2-9109acf1dc84 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:23.278611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf50233-4507-4117-aaf2-9109acf1dc84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:23.278611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf50233-4507-4117-aaf2-9109acf1dc84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:23.284491 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf50233-4507-4117-aaf2-9109acf1dc84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:23.285977 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf50233-4507-4117-aaf2-9109acf1dc84 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:23.297605 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1344/2722] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:00:25.177046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02f962-b5d1-486c-97a2-8f6ac1c7f734 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:25.177552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02f962-b5d1-486c-97a2-8f6ac1c7f734 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:25.177662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02f962-b5d1-486c-97a2-8f6ac1c7f734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:25.204391 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ed02f962-b5d1-486c-97a2-8f6ac1c7f734 None None] MFA Rules not processed for user `9d1076aed12f4d22a34df10ed0d8e462`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:25.205607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ed02f962-b5d1-486c-97a2-8f6ac1c7f734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:25.239675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:25.268700 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1379/2723] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:25.597504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf50233-4507-4117-aaf2-9109acf1dc84 service cinder] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:25.599291 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-0bf50233-4507-4117-aaf2-9109acf1dc84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:25.634873 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:25.666232 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf50233-4507-4117-aaf2-9109acf1dc84 service cinder] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:25.670619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0a34f-6954-40d7-afe3-24e33de74d13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:25.671308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0a34f-6954-40d7-afe3-24e33de74d13 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:25.671596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0a34f-6954-40d7-afe3-24e33de74d13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:25.671870 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0a34f-6954-40d7-afe3-24e33de74d13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:25.673666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2db0a34f-6954-40d7-afe3-24e33de74d13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:25.684989 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0a34f-6954-40d7-afe3-24e33de74d13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:25.686534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0a34f-6954-40d7-afe3-24e33de74d13 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:25.695956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1345/2724] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:00:35.376392 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=10 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:35.743353 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=15 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:53.063062 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02f962-b5d1-486c-97a2-8f6ac1c7f734 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:53.142300 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02f962-b5d1-486c-97a2-8f6ac1c7f734 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:53.148104 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58350610-c3ca-4936-a90e-9771384c85cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:53.148767 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58350610-c3ca-4936-a90e-9771384c85cf service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:53.149056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58350610-c3ca-4936-a90e-9771384c85cf service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:53.149339 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58350610-c3ca-4936-a90e-9771384c85cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:53.151214 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-58350610-c3ca-4936-a90e-9771384c85cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:53.166532 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58350610-c3ca-4936-a90e-9771384c85cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:53.168121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58350610-c3ca-4936-a90e-9771384c85cf service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:53.180082 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1380/2725] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:00:57.837806 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0a34f-6954-40d7-afe3-24e33de74d13 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:57.839221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2db0a34f-6954-40d7-afe3-24e33de74d13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:57.855368 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0a34f-6954-40d7-afe3-24e33de74d13 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:57.861899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb9367-64a8-46bd-8e62-9dd238f278b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:57.862531 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb9367-64a8-46bd-8e62-9dd238f278b8 service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:57.862855 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb9367-64a8-46bd-8e62-9dd238f278b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:57.863152 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb9367-64a8-46bd-8e62-9dd238f278b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:57.864884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-68bb9367-64a8-46bd-8e62-9dd238f278b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:57.880018 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb9367-64a8-46bd-8e62-9dd238f278b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:57.882093 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb9367-64a8-46bd-8e62-9dd238f278b8 service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:57.894003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1346/2726] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:00:58.728343 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58350610-c3ca-4936-a90e-9771384c85cf service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:58.808289 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58350610-c3ca-4936-a90e-9771384c85cf service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:58.814896 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780abdd0-6016-4721-be1a-bd3f33324a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:58.815648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780abdd0-6016-4721-be1a-bd3f33324a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:58.815999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780abdd0-6016-4721-be1a-bd3f33324a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:58.816346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780abdd0-6016-4721-be1a-bd3f33324a00 admin admin] PATH_INFO: `/v3/users/d7389e2ff6f042a58e6005412ff69c19` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:58.817720 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780abdd0-6016-4721-be1a-bd3f33324a00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7389e2ff6f042a58e6005412ff69c19)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:58.819850 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780abdd0-6016-4721-be1a-bd3f33324a00 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:59.035145 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-780abdd0-6016-4721-be1a-bd3f33324a00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7389e2ff6f042a58e6005412ff69c19'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:59.035792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-780abdd0-6016-4721-be1a-bd3f33324a00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7389e2ff6f042a58e6005412ff69c19'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:59.036392 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-780abdd0-6016-4721-be1a-bd3f33324a00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7389e2ff6f042a58e6005412ff69c19'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:59.113543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-780abdd0-6016-4721-be1a-bd3f33324a00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7389e2ff6f042a58e6005412ff69c19'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:59.121688 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-780abdd0-6016-4721-be1a-bd3f33324a00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7389e2ff6f042a58e6005412ff69c19'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:59.146179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1381/2727] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:00:58 2022] DELETE /identity/v3/users/d7389e2ff6f042a58e6005412ff69c19 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:59.150769 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb071ad-695e-4c10-b2cd-bcb60f67a696 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:59.151256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb071ad-695e-4c10-b2cd-bcb60f67a696 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:59.151363 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb071ad-695e-4c10-b2cd-bcb60f67a696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:59.170361 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-ceb071ad-695e-4c10-b2cd-bcb60f67a696 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:59.171551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ceb071ad-695e-4c10-b2cd-bcb60f67a696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:59.239813 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1347/2728] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:00:59.265019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780abdd0-6016-4721-be1a-bd3f33324a00 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:59.266597 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-780abdd0-6016-4721-be1a-bd3f33324a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:59.273419 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:59.331749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780abdd0-6016-4721-be1a-bd3f33324a00 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:59.336438 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efb400-3f0f-4dfd-bc72-cc0d36bcfcc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:59.337125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efb400-3f0f-4dfd-bc72-cc0d36bcfcc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:59.337406 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efb400-3f0f-4dfd-bc72-cc0d36bcfcc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:59.337681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efb400-3f0f-4dfd-bc72-cc0d36bcfcc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:59.348338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efb400-3f0f-4dfd-bc72-cc0d36bcfcc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:59.350081 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efb400-3f0f-4dfd-bc72-cc0d36bcfcc0 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:59.360954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1382/2729] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:00:59.765941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb071ad-695e-4c10-b2cd-bcb60f67a696 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:59.767420 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ceb071ad-695e-4c10-b2cd-bcb60f67a696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:59.779381 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb071ad-695e-4c10-b2cd-bcb60f67a696 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:59.796688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacfc27-0843-4086-ae9b-87f85a586d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:59.797523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacfc27-0843-4086-ae9b-87f85a586d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:59.797880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacfc27-0843-4086-ae9b-87f85a586d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:59.798223 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacfc27-0843-4086-ae9b-87f85a586d1b admin admin] PATH_INFO: `/v3/projects/a7a25e763a1b4d3b9ca1cb0deb04e365` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:59.799425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eacfc27-0843-4086-ae9b-87f85a586d1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7a25e763a1b4d3b9ca1cb0deb04e365)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:59.801470 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eacfc27-0843-4086-ae9b-87f85a586d1b admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:59.879186 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9eacfc27-0843-4086-ae9b-87f85a586d1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a25e763a1b4d3b9ca1cb0deb04e365', 'request_id': 'req-9eacfc27-0843-4086-ae9b-87f85a586d1b'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:59.885699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9eacfc27-0843-4086-ae9b-87f85a586d1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a25e763a1b4d3b9ca1cb0deb04e365', 'request_id': 'req-9eacfc27-0843-4086-ae9b-87f85a586d1b'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:59.891333 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9eacfc27-0843-4086-ae9b-87f85a586d1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a25e763a1b4d3b9ca1cb0deb04e365', 'request_id': 'req-9eacfc27-0843-4086-ae9b-87f85a586d1b'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:59.935817 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9eacfc27-0843-4086-ae9b-87f85a586d1b admin admin] The token cache is being invalidate because project a7a25e763a1b4d3b9ca1cb0deb04e365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:00:59.937907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9eacfc27-0843-4086-ae9b-87f85a586d1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:59.937907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-9eacfc27-0843-4086-ae9b-87f85a586d1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:59.938535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1348/2730] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:00:59 2022] DELETE /identity/v3/projects/a7a25e763a1b4d3b9ca1cb0deb04e365 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:59.943514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05d280-5ba5-4665-9412-580d5221c932 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:59.944008 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05d280-5ba5-4665-9412-580d5221c932 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:59.944127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05d280-5ba5-4665-9412-580d5221c932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:59.964064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5f05d280-5ba5-4665-9412-580d5221c932 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:00:59.965811 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5f05d280-5ba5-4665-9412-580d5221c932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:00.029965 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1383/2731] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:01:00.037672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96897d-dceb-4017-8aae-c4395301f93b None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:00.037963 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96897d-dceb-4017-8aae-c4395301f93b None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:00.038238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96897d-dceb-4017-8aae-c4395301f93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:00.058115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-2a96897d-dceb-4017-8aae-c4395301f93b None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:01:00.059350 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2a96897d-dceb-4017-8aae-c4395301f93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:00.067459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1349/2732] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:01:00.071921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05d280-5ba5-4665-9412-580d5221c932 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:00.082301 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05d280-5ba5-4665-9412-580d5221c932 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:00.086940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c09ae-fd3f-40da-9f40-d55b3d6bf498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:00.087547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c09ae-fd3f-40da-9f40-d55b3d6bf498 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:00.087828 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c09ae-fd3f-40da-9f40-d55b3d6bf498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:00.088106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c09ae-fd3f-40da-9f40-d55b3d6bf498 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:00.088807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c09ae-fd3f-40da-9f40-d55b3d6bf498 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:01:00.089101 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c09ae-fd3f-40da-9f40-d55b3d6bf498 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:00.091369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c09ae-fd3f-40da-9f40-d55b3d6bf498 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:00.102954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1384/2733] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 17:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:01:00.107414 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96897d-dceb-4017-8aae-c4395301f93b None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:00.117605 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96897d-dceb-4017-8aae-c4395301f93b None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:00.122364 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87fa27-c761-4600-b1ff-a79e2299c8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:00.122968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87fa27-c761-4600-b1ff-a79e2299c8e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:00.123248 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87fa27-c761-4600-b1ff-a79e2299c8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:00.123517 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87fa27-c761-4600-b1ff-a79e2299c8e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:00.124186 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87fa27-c761-4600-b1ff-a79e2299c8e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:00.126136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87fa27-c761-4600-b1ff-a79e2299c8e7 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:00.224252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1350/2734] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:01:00 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:01:00.227968 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c09ae-fd3f-40da-9f40-d55b3d6bf498 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:00.237949 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c09ae-fd3f-40da-9f40-d55b3d6bf498 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:00.296986 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817501c7-7605-498a-ad9d-50bda5a7e01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:00.297523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817501c7-7605-498a-ad9d-50bda5a7e01a admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:00.297807 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817501c7-7605-498a-ad9d-50bda5a7e01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:00.298083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817501c7-7605-498a-ad9d-50bda5a7e01a admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:00.298717 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817501c7-7605-498a-ad9d-50bda5a7e01a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:00.300680 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817501c7-7605-498a-ad9d-50bda5a7e01a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:00.388416 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1385/2735] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:01:00 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:01:00.392613 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87fa27-c761-4600-b1ff-a79e2299c8e7 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:00.402714 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87fa27-c761-4600-b1ff-a79e2299c8e7 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:00.408057 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e498a-c042-4ee0-aa63-84ae8a2bc725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:00.408687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e498a-c042-4ee0-aa63-84ae8a2bc725 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:00.408968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e498a-c042-4ee0-aa63-84ae8a2bc725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:00.409237 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e498a-c042-4ee0-aa63-84ae8a2bc725 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:00.409871 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e498a-c042-4ee0-aa63-84ae8a2bc725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:00.411456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e498a-c042-4ee0-aa63-84ae8a2bc725 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:00.421881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1351/2736] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:01:00.425806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817501c7-7605-498a-ad9d-50bda5a7e01a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:00.436141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817501c7-7605-498a-ad9d-50bda5a7e01a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:00.440692 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae3cdc-ebd9-4e39-9a0f-fe6b9493c562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:00.441279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae3cdc-ebd9-4e39-9a0f-fe6b9493c562 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:00.441557 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae3cdc-ebd9-4e39-9a0f-fe6b9493c562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:00.441829 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae3cdc-ebd9-4e39-9a0f-fe6b9493c562 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:00.442484 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae3cdc-ebd9-4e39-9a0f-fe6b9493c562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:00.444087 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae3cdc-ebd9-4e39-9a0f-fe6b9493c562 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:00.456218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1386/2737] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:01:00.460253 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e498a-c042-4ee0-aa63-84ae8a2bc725 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:00.470272 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e498a-c042-4ee0-aa63-84ae8a2bc725 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:00.474857 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef927fc6-611d-42aa-8a2a-abfa711fa314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:00.475496 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef927fc6-611d-42aa-8a2a-abfa711fa314 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:00.475782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef927fc6-611d-42aa-8a2a-abfa711fa314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:00.476061 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef927fc6-611d-42aa-8a2a-abfa711fa314 admin admin] PATH_INFO: `/v3/projects/6d4ca39653d24886b79e5e7175e7b4cc/users/4bdcfac8e4bf4b589d3958be4fd032f2/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:00.490409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef927fc6-611d-42aa-8a2a-abfa711fa314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d4ca39653d24886b79e5e7175e7b4cc, user_id=4bdcfac8e4bf4b589d3958be4fd032f2, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:00.493098 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef927fc6-611d-42aa-8a2a-abfa711fa314 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:00.547114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1352/2738] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:01:00 2022] PUT /identity/v3/projects/6d4ca39653d24886b79e5e7175e7b4cc/users/4bdcfac8e4bf4b589d3958be4fd032f2/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:01:00.552040 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b2d2f-f2c9-4805-b2d9-d6aefe7c2f17 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:00.552392 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b2d2f-f2c9-4805-b2d9-d6aefe7c2f17 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:00.552696 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b2d2f-f2c9-4805-b2d9-d6aefe7c2f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:00.571689 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8f0b2d2f-f2c9-4805-b2d9-d6aefe7c2f17 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:01:00.572908 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8f0b2d2f-f2c9-4805-b2d9-d6aefe7c2f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:00.581588 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:01:00.631008 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1387/2739] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:01:00.641339 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef927fc6-611d-42aa-8a2a-abfa711fa314 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:00.643041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ef927fc6-611d-42aa-8a2a-abfa711fa314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:00.652964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:01:00.699027 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef927fc6-611d-42aa-8a2a-abfa711fa314 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:00.703349 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfa219-f527-43ad-9d6a-75429d4e97e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:00.703944 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfa219-f527-43ad-9d6a-75429d4e97e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:00.704222 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfa219-f527-43ad-9d6a-75429d4e97e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:00.704528 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfa219-f527-43ad-9d6a-75429d4e97e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:00.713755 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becfa219-f527-43ad-9d6a-75429d4e97e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:00.715210 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becfa219-f527-43ad-9d6a-75429d4e97e8 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:00.724615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1353/2740] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:01:05.895911 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d76d74-557f-4490-9a56-681b1f8b6c66 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:05.897024 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d76d74-557f-4490-9a56-681b1f8b6c66 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:05.897556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d76d74-557f-4490-9a56-681b1f8b6c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:05.935251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b3d76d74-557f-4490-9a56-681b1f8b6c66 None None] MFA Rules not processed for user `4bdcfac8e4bf4b589d3958be4fd032f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:01:05.936589 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b3d76d74-557f-4490-9a56-681b1f8b6c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:06.008668 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1388/2741] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:01:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:01:06.013326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfa219-f527-43ad-9d6a-75429d4e97e8 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:06.023416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfa219-f527-43ad-9d6a-75429d4e97e8 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:06.029308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60d0d8-5686-424a-ab60-9443ea22a7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:06.029933 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60d0d8-5686-424a-ab60-9443ea22a7ef admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:06.030403 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60d0d8-5686-424a-ab60-9443ea22a7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:06.030845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60d0d8-5686-424a-ab60-9443ea22a7ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:06.031645 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b60d0d8-5686-424a-ab60-9443ea22a7ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:06.034207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b60d0d8-5686-424a-ab60-9443ea22a7ef admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:06.143576 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1354/2742] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:01:06 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:01:06.147621 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d76d74-557f-4490-9a56-681b1f8b6c66 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:06.159319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d76d74-557f-4490-9a56-681b1f8b6c66 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:06.238159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28c821-581e-4523-8755-2b025e1e0883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:06.238820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28c821-581e-4523-8755-2b025e1e0883 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:06.239099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28c821-581e-4523-8755-2b025e1e0883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:06.239372 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28c821-581e-4523-8755-2b025e1e0883 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:06.240063 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28c821-581e-4523-8755-2b025e1e0883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:06.244675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28c821-581e-4523-8755-2b025e1e0883 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:06.338033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1389/2743] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:01:06 2022] POST /identity/v3/users => generated 299 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:01:06.341034 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60d0d8-5686-424a-ab60-9443ea22a7ef admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:06.352640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60d0d8-5686-424a-ab60-9443ea22a7ef admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:06.357607 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30625d-ff3f-4280-9504-6053ac78e73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:06.358233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30625d-ff3f-4280-9504-6053ac78e73e admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:06.358518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30625d-ff3f-4280-9504-6053ac78e73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:06.358788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30625d-ff3f-4280-9504-6053ac78e73e admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:06.359438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30625d-ff3f-4280-9504-6053ac78e73e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:06.361071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30625d-ff3f-4280-9504-6053ac78e73e admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:06.372475 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1355/2744] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:01:06.376950 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28c821-581e-4523-8755-2b025e1e0883 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:06.387976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28c821-581e-4523-8755-2b025e1e0883 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:06.392750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b52e6-92c6-4d84-b7c7-c93c1a100bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:06.393416 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b52e6-92c6-4d84-b7c7-c93c1a100bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:06.393698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b52e6-92c6-4d84-b7c7-c93c1a100bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:06.393973 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b52e6-92c6-4d84-b7c7-c93c1a100bd5 admin admin] PATH_INFO: `/v3/projects/1bfe56ba3f204decb87551bae86e7eb6/users/75a4f8dde7af492c8eeffd0dfe5c904b/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:06.410510 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b52e6-92c6-4d84-b7c7-c93c1a100bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bfe56ba3f204decb87551bae86e7eb6, user_id=75a4f8dde7af492c8eeffd0dfe5c904b, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:06.413344 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b52e6-92c6-4d84-b7c7-c93c1a100bd5 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:06.507817 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1390/2745] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:01:06 2022] PUT /identity/v3/projects/1bfe56ba3f204decb87551bae86e7eb6/users/75a4f8dde7af492c8eeffd0dfe5c904b/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:01:06.511825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30625d-ff3f-4280-9504-6053ac78e73e admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:06.522897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30625d-ff3f-4280-9504-6053ac78e73e admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:06.587744 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a3c10-4ceb-452a-b429-eee432a7ee0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:06.588367 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a3c10-4ceb-452a-b429-eee432a7ee0f admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:06.588679 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a3c10-4ceb-452a-b429-eee432a7ee0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:06.588949 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a3c10-4ceb-452a-b429-eee432a7ee0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:06.589698 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a3c10-4ceb-452a-b429-eee432a7ee0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:06.591338 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a3c10-4ceb-452a-b429-eee432a7ee0f admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:06.602228 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1356/2746] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:01:06.606395 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b52e6-92c6-4d84-b7c7-c93c1a100bd5 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:06.616903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b52e6-92c6-4d84-b7c7-c93c1a100bd5 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:06.622188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9907ce5-d79c-4f37-ae6a-aae51e5ac82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:06.622857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9907ce5-d79c-4f37-ae6a-aae51e5ac82c admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:06.623138 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9907ce5-d79c-4f37-ae6a-aae51e5ac82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:06.623416 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9907ce5-d79c-4f37-ae6a-aae51e5ac82c admin admin] PATH_INFO: `/v3/projects/1bfe56ba3f204decb87551bae86e7eb6/users/75a4f8dde7af492c8eeffd0dfe5c904b/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:06.625396 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9907ce5-d79c-4f37-ae6a-aae51e5ac82c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bfe56ba3f204decb87551bae86e7eb6, user_id=75a4f8dde7af492c8eeffd0dfe5c904b, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:06.628052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9907ce5-d79c-4f37-ae6a-aae51e5ac82c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:06.631883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:01:06.640556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1391/2747] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:01:06 2022] PUT /identity/v3/projects/1bfe56ba3f204decb87551bae86e7eb6/users/75a4f8dde7af492c8eeffd0dfe5c904b/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:01:12.111771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850d03d-562e-402b-8641-5d5415be6956 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:12.112884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850d03d-562e-402b-8641-5d5415be6956 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:12.113458 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850d03d-562e-402b-8641-5d5415be6956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:12.143410 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-5850d03d-562e-402b-8641-5d5415be6956 None None] MFA Rules not processed for user `75a4f8dde7af492c8eeffd0dfe5c904b`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:01:12.145116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-5850d03d-562e-402b-8641-5d5415be6956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:12.214014 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:01:12.220548 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1357/2748] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:01:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:01:12.235071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9907ce5-d79c-4f37-ae6a-aae51e5ac82c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:12.237649 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f9907ce5-d79c-4f37-ae6a-aae51e5ac82c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:12.321486 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9907ce5-d79c-4f37-ae6a-aae51e5ac82c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:12.325838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acc838-87c7-47f6-b067-279e223db46e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56418ff737504181a59714aad9f1c375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:12.326445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acc838-87c7-47f6-b067-279e223db46e service glance] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:12.326724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acc838-87c7-47f6-b067-279e223db46e service glance] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:12.326994 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acc838-87c7-47f6-b067-279e223db46e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:12.388569 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28acc838-87c7-47f6-b067-279e223db46e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:12.391069 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28acc838-87c7-47f6-b067-279e223db46e service glance] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:12.401266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1392/2749] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:01:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:01:12.805066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e115878-ddc9-4702-9ddb-97cfde896d02 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:12.805493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e115878-ddc9-4702-9ddb-97cfde896d02 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:12.805779 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e115878-ddc9-4702-9ddb-97cfde896d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:12.828444 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-6e115878-ddc9-4702-9ddb-97cfde896d02 None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:01:12.829588 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6e115878-ddc9-4702-9ddb-97cfde896d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:12.892720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1358/2750] 10.222.0.70 () {58 vars in 1009 bytes} [Wed Aug 24 17:01:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:01:13.084654 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acc838-87c7-47f6-b067-279e223db46e service glance] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:13.086116 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-28acc838-87c7-47f6-b067-279e223db46e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:13.207592 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acc838-87c7-47f6-b067-279e223db46e service glance] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:13.213300 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a106ff-9d33-4e88-aecc-4113a58f6685 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:13.216099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a106ff-9d33-4e88-aecc-4113a58f6685 service swift] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:13.216099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a106ff-9d33-4e88-aecc-4113a58f6685 service swift] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:13.216099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a106ff-9d33-4e88-aecc-4113a58f6685 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:13.224213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a106ff-9d33-4e88-aecc-4113a58f6685 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:13.225838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a106ff-9d33-4e88-aecc-4113a58f6685 service swift] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:13.238927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1393/2751] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 17:01:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:01:18.836196 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e115878-ddc9-4702-9ddb-97cfde896d02 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:18.837723 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6e115878-ddc9-4702-9ddb-97cfde896d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:18.937092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e115878-ddc9-4702-9ddb-97cfde896d02 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:18.942953 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e15b2-e7c0-433e-b51c-a1f78cc9b40e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:18.943776 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e15b2-e7c0-433e-b51c-a1f78cc9b40e service placement] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:18.944071 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e15b2-e7c0-433e-b51c-a1f78cc9b40e service placement] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:18.944347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e15b2-e7c0-433e-b51c-a1f78cc9b40e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:18.947154 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-906e15b2-e7c0-433e-b51c-a1f78cc9b40e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:18.966078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e15b2-e7c0-433e-b51c-a1f78cc9b40e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:18.967739 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e15b2-e7c0-433e-b51c-a1f78cc9b40e service placement] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:18.980640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1359/2752] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:01:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:01:23.231146 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:01:23.826624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35349f18-5e93-475a-befa-57135e8b0245 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:23.826917 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35349f18-5e93-475a-befa-57135e8b0245 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:23.827191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35349f18-5e93-475a-befa-57135e8b0245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:23.846858 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-35349f18-5e93-475a-befa-57135e8b0245 None None] MFA Rules not processed for user `4bdcfac8e4bf4b589d3958be4fd032f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:01:23.848076 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-35349f18-5e93-475a-befa-57135e8b0245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:23.855358 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1394/2753] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:01:27.250048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf790a6-cdcc-4fb0-9210-709648024881 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:27.250558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf790a6-cdcc-4fb0-9210-709648024881 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:27.250734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf790a6-cdcc-4fb0-9210-709648024881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:27.270236 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-9cf790a6-cdcc-4fb0-9210-709648024881 None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:01:27.274911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-9cf790a6-cdcc-4fb0-9210-709648024881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:27.278884 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1360/2754] 10.222.0.70 () {58 vars in 1009 bytes} [Wed Aug 24 17:01:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:01:27.291632 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35349f18-5e93-475a-befa-57135e8b0245 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:27.304757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35349f18-5e93-475a-befa-57135e8b0245 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:27.310358 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea6aea-7b51-44f0-86e3-2655d3457bde service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:27.311003 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea6aea-7b51-44f0-86e3-2655d3457bde service swift] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:27.311258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea6aea-7b51-44f0-86e3-2655d3457bde service swift] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:27.311529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea6aea-7b51-44f0-86e3-2655d3457bde service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:27.319820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea6aea-7b51-44f0-86e3-2655d3457bde service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:27.321628 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea6aea-7b51-44f0-86e3-2655d3457bde service swift] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:27.330303 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1395/2755] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 17:01:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:01:31.054197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf790a6-cdcc-4fb0-9210-709648024881 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:31.056581 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-9cf790a6-cdcc-4fb0-9210-709648024881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:31.076740 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf790a6-cdcc-4fb0-9210-709648024881 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:31.082058 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fcae5-3163-4b47-94fd-94afe692fc4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:31.082728 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fcae5-3163-4b47-94fd-94afe692fc4d service placement] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:31.082964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fcae5-3163-4b47-94fd-94afe692fc4d service placement] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:31.083235 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fcae5-3163-4b47-94fd-94afe692fc4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:31.085248 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-895fcae5-3163-4b47-94fd-94afe692fc4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:31.098206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895fcae5-3163-4b47-94fd-94afe692fc4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:31.100016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895fcae5-3163-4b47-94fd-94afe692fc4d service placement] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:31.112123 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1361/2756] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:01:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:01:31.200748 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea6aea-7b51-44f0-86e3-2655d3457bde service swift] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:31.202509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2aea6aea-7b51-44f0-86e3-2655d3457bde service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:01:31.274522 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea6aea-7b51-44f0-86e3-2655d3457bde service swift] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:31.279796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ee61e-06d8-424c-8a4f-70f206ee58b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:31.280545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ee61e-06d8-424c-8a4f-70f206ee58b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:31.280905 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ee61e-06d8-424c-8a4f-70f206ee58b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:31.281250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ee61e-06d8-424c-8a4f-70f206ee58b0 admin admin] PATH_INFO: `/v3/users/d715cc7828f847529f795f356a8ac16b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:31.282482 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ee61e-06d8-424c-8a4f-70f206ee58b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d715cc7828f847529f795f356a8ac16b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:31.284432 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ee61e-06d8-424c-8a4f-70f206ee58b0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:31.482052 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-973ee61e-06d8-424c-8a4f-70f206ee58b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd715cc7828f847529f795f356a8ac16b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:01:31.482868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-973ee61e-06d8-424c-8a4f-70f206ee58b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd715cc7828f847529f795f356a8ac16b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:01:31.483529 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-973ee61e-06d8-424c-8a4f-70f206ee58b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd715cc7828f847529f795f356a8ac16b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:01:31.565409 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-973ee61e-06d8-424c-8a4f-70f206ee58b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd715cc7828f847529f795f356a8ac16b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:01:31.576910 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-973ee61e-06d8-424c-8a4f-70f206ee58b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd715cc7828f847529f795f356a8ac16b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:01:31.596369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1396/2757] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:01:31 2022] DELETE /identity/v3/users/d715cc7828f847529f795f356a8ac16b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:01:31.874513 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fcae5-3163-4b47-94fd-94afe692fc4d service placement] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:01:31.886599 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fcae5-3163-4b47-94fd-94afe692fc4d service placement] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:01:31.961812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592db2af-6bc9-41af-b500-77e6a33a3b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:01:31.962493 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592db2af-6bc9-41af-b500-77e6a33a3b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:01:31.962770 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592db2af-6bc9-41af-b500-77e6a33a3b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:01:31.963044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592db2af-6bc9-41af-b500-77e6a33a3b0a admin admin] PATH_INFO: `/v3/projects/1088b1d454684ca5be800a99d4e26304` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:01:31.964328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592db2af-6bc9-41af-b500-77e6a33a3b0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1088b1d454684ca5be800a99d4e26304)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:01:31.966368 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592db2af-6bc9-41af-b500-77e6a33a3b0a admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:01:31.992304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:01:32.608162 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-592db2af-6bc9-41af-b500-77e6a33a3b0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1088b1d454684ca5be800a99d4e26304', 'request_id': 'req-592db2af-6bc9-41af-b500-77e6a33a3b0a'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:01:32.618920 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-592db2af-6bc9-41af-b500-77e6a33a3b0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1088b1d454684ca5be800a99d4e26304', 'request_id': 'req-592db2af-6bc9-41af-b500-77e6a33a3b0a'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:01:32.623048 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-592db2af-6bc9-41af-b500-77e6a33a3b0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1088b1d454684ca5be800a99d4e26304', 'request_id': 'req-592db2af-6bc9-41af-b500-77e6a33a3b0a'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:01:32.713353 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-592db2af-6bc9-41af-b500-77e6a33a3b0a admin admin] The token cache is being invalidate because project 1088b1d454684ca5be800a99d4e26304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:01:32.727394 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-592db2af-6bc9-41af-b500-77e6a33a3b0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:01:32.727394 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-592db2af-6bc9-41af-b500-77e6a33a3b0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:01:32.727394 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1362/2758] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:01:31 2022] DELETE /identity/v3/projects/1088b1d454684ca5be800a99d4e26304 => generated 0 bytes in 843 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:01:41.596687 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=62,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:01:42.709008 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=18,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:02:02.200250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ee61e-06d8-424c-8a4f-70f206ee58b0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:02:02.202251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-973ee61e-06d8-424c-8a4f-70f206ee58b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:02:02.257616 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ee61e-06d8-424c-8a4f-70f206ee58b0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:02:02.303829 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614715a-d4cd-46ed-9584-1181de989cab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:02:02.304514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614715a-d4cd-46ed-9584-1181de989cab service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:02:02.304799 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614715a-d4cd-46ed-9584-1181de989cab service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:02:02.305068 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614715a-d4cd-46ed-9584-1181de989cab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:02:02.306768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3614715a-d4cd-46ed-9584-1181de989cab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:02:02.366162 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614715a-d4cd-46ed-9584-1181de989cab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:02:02.367560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614715a-d4cd-46ed-9584-1181de989cab service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:02:02.376942 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1397/2759] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:02:02 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:02:12.373127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=70 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:02:44.544830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4f830-cd4f-4e3b-b90f-82c477d417b2 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:02:44.546026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4f830-cd4f-4e3b-b90f-82c477d417b2 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:02:44.546157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4f830-cd4f-4e3b-b90f-82c477d417b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:02:44.565151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-19a4f830-cd4f-4e3b-b90f-82c477d417b2 None None] MFA Rules not processed for user `701be5a3cff34016ab22b6dc4d23d939`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:02:44.566348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-19a4f830-cd4f-4e3b-b90f-82c477d417b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:02:44.620182 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1363/2760] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:02:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:02:54.613695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=25 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:02:55.680338 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614715a-d4cd-46ed-9584-1181de989cab service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:02:55.681756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3614715a-d4cd-46ed-9584-1181de989cab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:02:55.693068 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614715a-d4cd-46ed-9584-1181de989cab service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:02:55.696741 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cf541-5122-43b4-88a5-1598471b0c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:02:55.697340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cf541-5122-43b4-88a5-1598471b0c76 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:02:55.697618 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cf541-5122-43b4-88a5-1598471b0c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:02:55.697888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cf541-5122-43b4-88a5-1598471b0c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:02:55.699533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-636cf541-5122-43b4-88a5-1598471b0c76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:02:55.768822 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cf541-5122-43b4-88a5-1598471b0c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:02:55.770272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cf541-5122-43b4-88a5-1598471b0c76 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:02:55.781444 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1398/2761] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:02:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:02:58.340778 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4f830-cd4f-4e3b-b90f-82c477d417b2 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:02:58.350369 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4f830-cd4f-4e3b-b90f-82c477d417b2 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:02:58.353891 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac642e2-6b48-4754-9de6-e9ff17cf3873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:02:58.354479 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac642e2-6b48-4754-9de6-e9ff17cf3873 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:02:58.354764 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac642e2-6b48-4754-9de6-e9ff17cf3873 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:02:58.355033 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac642e2-6b48-4754-9de6-e9ff17cf3873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:02:58.356757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-fac642e2-6b48-4754-9de6-e9ff17cf3873 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:02:58.369741 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac642e2-6b48-4754-9de6-e9ff17cf3873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:02:58.371150 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac642e2-6b48-4754-9de6-e9ff17cf3873 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:02:58.382656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1364/2762] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:02:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:03:05.776180 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=34 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:03:08.377684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=6 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:04:12.159405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cf541-5122-43b4-88a5-1598471b0c76 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:04:12.160846 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-636cf541-5122-43b4-88a5-1598471b0c76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:04:12.171244 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cf541-5122-43b4-88a5-1598471b0c76 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:04:12.174754 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f183d-2176-4eef-b9b3-b61ec5811dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:04:12.175342 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f183d-2176-4eef-b9b3-b61ec5811dee service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:04:12.175619 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f183d-2176-4eef-b9b3-b61ec5811dee service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:04:12.175888 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f183d-2176-4eef-b9b3-b61ec5811dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:04:12.177541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-953f183d-2176-4eef-b9b3-b61ec5811dee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:04:12.188625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f183d-2176-4eef-b9b3-b61ec5811dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:04:12.190083 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f183d-2176-4eef-b9b3-b61ec5811dee service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:04:12.197882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1399/2763] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:04:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:04:22.194436 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=6 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:04:46.610816 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327d646-67b4-4b8b-8663-e84c97925728 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:04:46.611300 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327d646-67b4-4b8b-8663-e84c97925728 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:04:46.611562 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327d646-67b4-4b8b-8663-e84c97925728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:04:46.630572 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-0327d646-67b4-4b8b-8663-e84c97925728 None None] MFA Rules not processed for user `7f3899f2eb554f2fb5a7c43f660b51d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:04:46.631763 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-0327d646-67b4-4b8b-8663-e84c97925728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:04:46.685438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1365/2764] 10.222.0.70 () {58 vars in 967 bytes} [Wed Aug 24 17:04:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:04:46.725636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f183d-2176-4eef-b9b3-b61ec5811dee service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:04:46.727058 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-953f183d-2176-4eef-b9b3-b61ec5811dee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:04:46.797203 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f183d-2176-4eef-b9b3-b61ec5811dee service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:04:46.805258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3bf74-6e5c-4ef7-8346-ded368688c7d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8ec0b1e5f4d434bb725297f3f867ba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:04:46.805883 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3bf74-6e5c-4ef7-8346-ded368688c7d service swift] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:04:46.806277 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3bf74-6e5c-4ef7-8346-ded368688c7d service swift] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:04:46.806552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3bf74-6e5c-4ef7-8346-ded368688c7d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:04:46.815775 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3bf74-6e5c-4ef7-8346-ded368688c7d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:04:46.817386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3bf74-6e5c-4ef7-8346-ded368688c7d service swift] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:04:46.825672 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1400/2765] 10.222.0.70 () {60 vars in 1384 bytes} [Wed Aug 24 17:04:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:04:51.128852 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327d646-67b4-4b8b-8663-e84c97925728 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:04:51.130899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-0327d646-67b4-4b8b-8663-e84c97925728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:04:51.201099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327d646-67b4-4b8b-8663-e84c97925728 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:04:51.206064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8c208-2ab4-4a63-a5ad-3d2ed239048e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:04:51.206665 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8c208-2ab4-4a63-a5ad-3d2ed239048e service placement] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:04:51.206945 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8c208-2ab4-4a63-a5ad-3d2ed239048e service placement] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:04:51.207218 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8c208-2ab4-4a63-a5ad-3d2ed239048e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:04:51.209072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2ef8c208-2ab4-4a63-a5ad-3d2ed239048e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:04:51.221012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8c208-2ab4-4a63-a5ad-3d2ed239048e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:04:51.222982 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8c208-2ab4-4a63-a5ad-3d2ed239048e service placement] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:04:51.238099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1366/2766] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:04:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:04:54.832635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3bf74-6e5c-4ef7-8346-ded368688c7d service swift] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:04:54.834213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-71d3bf74-6e5c-4ef7-8346-ded368688c7d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:04:54.898397 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3bf74-6e5c-4ef7-8346-ded368688c7d service swift] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:04:54.904972 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c19478-b9e8-423d-a346-bd31d692cd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:04:54.905630 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c19478-b9e8-423d-a346-bd31d692cd5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:04:54.905912 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c19478-b9e8-423d-a346-bd31d692cd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:04:54.906188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c19478-b9e8-423d-a346-bd31d692cd5a admin admin] PATH_INFO: `/v3/users/4bdcfac8e4bf4b589d3958be4fd032f2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:04:54.907323 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c19478-b9e8-423d-a346-bd31d692cd5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bdcfac8e4bf4b589d3958be4fd032f2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:04:54.909279 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c19478-b9e8-423d-a346-bd31d692cd5a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:04:55.079739 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-34c19478-b9e8-423d-a346-bd31d692cd5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bdcfac8e4bf4b589d3958be4fd032f2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:55.080313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-34c19478-b9e8-423d-a346-bd31d692cd5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bdcfac8e4bf4b589d3958be4fd032f2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:55.080954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-34c19478-b9e8-423d-a346-bd31d692cd5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bdcfac8e4bf4b589d3958be4fd032f2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:55.157514 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-34c19478-b9e8-423d-a346-bd31d692cd5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bdcfac8e4bf4b589d3958be4fd032f2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:55.164796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-34c19478-b9e8-423d-a346-bd31d692cd5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bdcfac8e4bf4b589d3958be4fd032f2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:55.182935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1401/2767] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:04:54 2022] DELETE /identity/v3/users/4bdcfac8e4bf4b589d3958be4fd032f2 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:04:55.411587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8c208-2ab4-4a63-a5ad-3d2ed239048e service placement] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:04:55.422526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8c208-2ab4-4a63-a5ad-3d2ed239048e service placement] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:04:55.481310 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca7595-23bf-4bbe-a713-7e41b82d627e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:04:55.481967 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca7595-23bf-4bbe-a713-7e41b82d627e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:04:55.482252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca7595-23bf-4bbe-a713-7e41b82d627e admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:04:55.482527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca7595-23bf-4bbe-a713-7e41b82d627e admin admin] PATH_INFO: `/v3/projects/6d4ca39653d24886b79e5e7175e7b4cc` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:04:55.483671 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca7595-23bf-4bbe-a713-7e41b82d627e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d4ca39653d24886b79e5e7175e7b4cc)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:04:55.485624 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca7595-23bf-4bbe-a713-7e41b82d627e admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:04:55.578960 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:04:55.587131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-16ca7595-23bf-4bbe-a713-7e41b82d627e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d4ca39653d24886b79e5e7175e7b4cc', 'request_id': 'req-16ca7595-23bf-4bbe-a713-7e41b82d627e'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:55.593710 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-16ca7595-23bf-4bbe-a713-7e41b82d627e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d4ca39653d24886b79e5e7175e7b4cc', 'request_id': 'req-16ca7595-23bf-4bbe-a713-7e41b82d627e'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:55.599079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-16ca7595-23bf-4bbe-a713-7e41b82d627e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d4ca39653d24886b79e5e7175e7b4cc', 'request_id': 'req-16ca7595-23bf-4bbe-a713-7e41b82d627e'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:55.682946 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-16ca7595-23bf-4bbe-a713-7e41b82d627e admin admin] The token cache is being invalidate because project 6d4ca39653d24886b79e5e7175e7b4cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:04:55.683170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-16ca7595-23bf-4bbe-a713-7e41b82d627e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:55.683787 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-16ca7595-23bf-4bbe-a713-7e41b82d627e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:55.685311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1367/2768] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:04:55 2022] DELETE /identity/v3/projects/6d4ca39653d24886b79e5e7175e7b4cc => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:04:55.689913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c19478-b9e8-423d-a346-bd31d692cd5a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:04:55.691320 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-34c19478-b9e8-423d-a346-bd31d692cd5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:04:55.704066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c19478-b9e8-423d-a346-bd31d692cd5a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:04:55.708791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:04:55.761729 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e7f5b-6b3e-45e9-8963-5c7c607fbaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:04:55.762375 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e7f5b-6b3e-45e9-8963-5c7c607fbaf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:04:55.762662 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e7f5b-6b3e-45e9-8963-5c7c607fbaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:04:55.762942 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e7f5b-6b3e-45e9-8963-5c7c607fbaf5 admin admin] PATH_INFO: `/v3/users/75a4f8dde7af492c8eeffd0dfe5c904b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:04:55.764075 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e7f5b-6b3e-45e9-8963-5c7c607fbaf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a4f8dde7af492c8eeffd0dfe5c904b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:04:55.766093 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e7f5b-6b3e-45e9-8963-5c7c607fbaf5 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:04:55.948025 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-397e7f5b-6b3e-45e9-8963-5c7c607fbaf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a4f8dde7af492c8eeffd0dfe5c904b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:55.948674 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-397e7f5b-6b3e-45e9-8963-5c7c607fbaf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a4f8dde7af492c8eeffd0dfe5c904b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:55.949210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-397e7f5b-6b3e-45e9-8963-5c7c607fbaf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a4f8dde7af492c8eeffd0dfe5c904b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:56.049518 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-397e7f5b-6b3e-45e9-8963-5c7c607fbaf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a4f8dde7af492c8eeffd0dfe5c904b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:56.056715 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-397e7f5b-6b3e-45e9-8963-5c7c607fbaf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a4f8dde7af492c8eeffd0dfe5c904b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:56.073607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1402/2769] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:04:55 2022] DELETE /identity/v3/users/75a4f8dde7af492c8eeffd0dfe5c904b => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:04:56.320021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca7595-23bf-4bbe-a713-7e41b82d627e admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:04:56.321606 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-16ca7595-23bf-4bbe-a713-7e41b82d627e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:04:56.336252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca7595-23bf-4bbe-a713-7e41b82d627e admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:04:56.392933 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08297129-c058-41d4-b194-596c8d935e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:04:56.393564 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08297129-c058-41d4-b194-596c8d935e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:04:56.393844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08297129-c058-41d4-b194-596c8d935e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:04:56.394118 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08297129-c058-41d4-b194-596c8d935e01 admin admin] PATH_INFO: `/v3/projects/1bfe56ba3f204decb87551bae86e7eb6` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:04:56.395229 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08297129-c058-41d4-b194-596c8d935e01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bfe56ba3f204decb87551bae86e7eb6)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:04:56.397219 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08297129-c058-41d4-b194-596c8d935e01 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:04:56.517372 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-08297129-c058-41d4-b194-596c8d935e01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bfe56ba3f204decb87551bae86e7eb6', 'request_id': 'req-08297129-c058-41d4-b194-596c8d935e01'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:56.523384 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-08297129-c058-41d4-b194-596c8d935e01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bfe56ba3f204decb87551bae86e7eb6', 'request_id': 'req-08297129-c058-41d4-b194-596c8d935e01'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:56.528854 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-08297129-c058-41d4-b194-596c8d935e01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bfe56ba3f204decb87551bae86e7eb6', 'request_id': 'req-08297129-c058-41d4-b194-596c8d935e01'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:56.602790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-08297129-c058-41d4-b194-596c8d935e01 admin admin] The token cache is being invalidate because project 1bfe56ba3f204decb87551bae86e7eb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:04:56.603082 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-08297129-c058-41d4-b194-596c8d935e01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:56.603711 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-08297129-c058-41d4-b194-596c8d935e01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:04:56.605251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1368/2770] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:04:56 2022] DELETE /identity/v3/projects/1bfe56ba3f204decb87551bae86e7eb6 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:05:06.079121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:05:06.599730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:05:07.257594 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e7f5b-6b3e-45e9-8963-5c7c607fbaf5 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:05:07.259014 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-397e7f5b-6b3e-45e9-8963-5c7c607fbaf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:05:07.315290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e7f5b-6b3e-45e9-8963-5c7c607fbaf5 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:05:07.363740 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177567b-1cc6-4f07-bd01-60822090476e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:05:07.364346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177567b-1cc6-4f07-bd01-60822090476e service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:05:07.364656 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177567b-1cc6-4f07-bd01-60822090476e service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:05:07.364932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177567b-1cc6-4f07-bd01-60822090476e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:05:07.366663 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2177567b-1cc6-4f07-bd01-60822090476e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:05:07.424803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177567b-1cc6-4f07-bd01-60822090476e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:05:07.426971 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177567b-1cc6-4f07-bd01-60822090476e service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:05:07.436740 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1403/2771] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:05:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:05:17.432249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=68 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:05:20.271816 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddae10f-7b5f-4f56-842c-53a5e25a83c8 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:05:20.272308 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddae10f-7b5f-4f56-842c-53a5e25a83c8 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:05:20.272459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddae10f-7b5f-4f56-842c-53a5e25a83c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:05:20.290883 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-4ddae10f-7b5f-4f56-842c-53a5e25a83c8 None None] MFA Rules not processed for user `701be5a3cff34016ab22b6dc4d23d939`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:05:20.292068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-4ddae10f-7b5f-4f56-842c-53a5e25a83c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:05:20.298854 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1369/2772] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:05:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:05:20.605731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177567b-1cc6-4f07-bd01-60822090476e service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:05:20.607245 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2177567b-1cc6-4f07-bd01-60822090476e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:05:20.663327 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177567b-1cc6-4f07-bd01-60822090476e service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:05:20.666929 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be6853-136e-4aae-afd2-435824ea1750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:05:20.667533 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be6853-136e-4aae-afd2-435824ea1750 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:05:20.667815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be6853-136e-4aae-afd2-435824ea1750 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:05:20.668090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be6853-136e-4aae-afd2-435824ea1750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:05:20.669868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a8be6853-136e-4aae-afd2-435824ea1750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:05:20.680100 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be6853-136e-4aae-afd2-435824ea1750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:05:20.681581 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be6853-136e-4aae-afd2-435824ea1750 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:05:20.689768 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1404/2773] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:05:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:05:21.378348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddae10f-7b5f-4f56-842c-53a5e25a83c8 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:05:21.380809 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-4ddae10f-7b5f-4f56-842c-53a5e25a83c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:05:21.455654 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddae10f-7b5f-4f56-842c-53a5e25a83c8 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:05:21.460687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e16752-2ed6-4cc6-a748-3bbe33c5e4e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:05:21.461285 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16752-2ed6-4cc6-a748-3bbe33c5e4e3 service placement] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:05:21.461566 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16752-2ed6-4cc6-a748-3bbe33c5e4e3 service placement] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:05:21.461837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16752-2ed6-4cc6-a748-3bbe33c5e4e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:05:21.463567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a5e16752-2ed6-4cc6-a748-3bbe33c5e4e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:05:21.474984 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e16752-2ed6-4cc6-a748-3bbe33c5e4e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:05:21.476529 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e16752-2ed6-4cc6-a748-3bbe33c5e4e3 service placement] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:05:21.486201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1370/2774] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:05:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:05:22.417517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be6853-136e-4aae-afd2-435824ea1750 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:05:22.419266 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a8be6853-136e-4aae-afd2-435824ea1750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:05:22.514979 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be6853-136e-4aae-afd2-435824ea1750 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:05:22.523220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec6b09-b7ad-4405-ba59-a960f1fc09f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:05:22.539437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec6b09-b7ad-4405-ba59-a960f1fc09f3 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:05:22.539437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec6b09-b7ad-4405-ba59-a960f1fc09f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:05:22.539437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec6b09-b7ad-4405-ba59-a960f1fc09f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:05:22.539437 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d1ec6b09-b7ad-4405-ba59-a960f1fc09f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:05:22.616749 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec6b09-b7ad-4405-ba59-a960f1fc09f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:05:22.619410 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec6b09-b7ad-4405-ba59-a960f1fc09f3 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:05:22.637560 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1405/2775] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:05:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:05:22.760789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e16752-2ed6-4cc6-a748-3bbe33c5e4e3 service placement] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:05:22.763496 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a5e16752-2ed6-4cc6-a748-3bbe33c5e4e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:05:22.780219 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e16752-2ed6-4cc6-a748-3bbe33c5e4e3 service placement] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:05:22.786238 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3caf79-6dfe-490b-9974-7db8842c7b0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:05:22.786848 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3caf79-6dfe-490b-9974-7db8842c7b0e service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:05:22.787131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3caf79-6dfe-490b-9974-7db8842c7b0e service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:05:22.787558 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3caf79-6dfe-490b-9974-7db8842c7b0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:05:22.789399 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-0f3caf79-6dfe-490b-9974-7db8842c7b0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:05:22.802907 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3caf79-6dfe-490b-9974-7db8842c7b0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:05:22.804577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3caf79-6dfe-490b-9974-7db8842c7b0e service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:05:22.816932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1371/2776] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:05:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:05:28.176076 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141269cb-6c29-421b-adbd-1836f84c39be None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:05:28.176551 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141269cb-6c29-421b-adbd-1836f84c39be None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:05:28.176677 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141269cb-6c29-421b-adbd-1836f84c39be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:05:28.196155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-141269cb-6c29-421b-adbd-1836f84c39be None None] MFA Rules not processed for user `9d1076aed12f4d22a34df10ed0d8e462`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:05:28.197393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-141269cb-6c29-421b-adbd-1836f84c39be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:05:28.204210 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1406/2777] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:05:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:05:28.564317 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3caf79-6dfe-490b-9974-7db8842c7b0e service nova] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:05:28.574442 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3caf79-6dfe-490b-9974-7db8842c7b0e service nova] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:05:28.578148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91970b-4d23-4c30-9500-a68f60034de2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:05:28.578760 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91970b-4d23-4c30-9500-a68f60034de2 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:05:28.579017 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91970b-4d23-4c30-9500-a68f60034de2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:05:28.579288 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91970b-4d23-4c30-9500-a68f60034de2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:05:28.581049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2f91970b-4d23-4c30-9500-a68f60034de2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:05:28.590518 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91970b-4d23-4c30-9500-a68f60034de2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:05:28.591936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91970b-4d23-4c30-9500-a68f60034de2 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:05:28.600328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1372/2778] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:05:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:05:38.190025 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=85 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:05:38.599448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=47 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:05:41.741169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141269cb-6c29-421b-adbd-1836f84c39be None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:05:41.742648 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-141269cb-6c29-421b-adbd-1836f84c39be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:05:41.755776 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141269cb-6c29-421b-adbd-1836f84c39be None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:05:41.759349 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c2368-fef4-4e41-8c53-2e36f961fc61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:05:41.760067 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c2368-fef4-4e41-8c53-2e36f961fc61 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:05:41.760448 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c2368-fef4-4e41-8c53-2e36f961fc61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:05:41.762024 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c2368-fef4-4e41-8c53-2e36f961fc61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:05:41.763882 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b55c2368-fef4-4e41-8c53-2e36f961fc61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:05:41.830370 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c2368-fef4-4e41-8c53-2e36f961fc61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:05:41.831803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c2368-fef4-4e41-8c53-2e36f961fc61 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:05:41.841115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1407/2779] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:05:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:05:51.836819 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=30 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:20.115774 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91970b-4d23-4c30-9500-a68f60034de2 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:20.117207 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2f91970b-4d23-4c30-9500-a68f60034de2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:06:20.201103 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91970b-4d23-4c30-9500-a68f60034de2 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:20.206388 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6d90e-5a0c-4f51-a53c-572ebfad3939 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:20.207021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6d90e-5a0c-4f51-a53c-572ebfad3939 service placement] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:20.207304 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6d90e-5a0c-4f51-a53c-572ebfad3939 service placement] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:20.207578 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6d90e-5a0c-4f51-a53c-572ebfad3939 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:20.209476 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-30c6d90e-5a0c-4f51-a53c-572ebfad3939 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:06:20.220997 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6d90e-5a0c-4f51-a53c-572ebfad3939 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:20.223113 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6d90e-5a0c-4f51-a53c-572ebfad3939 service placement] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:20.232830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1373/2780] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:06:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:06:30.228789 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=37 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:46.943102 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c2368-fef4-4e41-8c53-2e36f961fc61 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:46.971498 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c2368-fef4-4e41-8c53-2e36f961fc61 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:46.975217 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48313dd8-16b9-4f56-b323-3e73e0621cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:46.975823 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48313dd8-16b9-4f56-b323-3e73e0621cac service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:46.976106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48313dd8-16b9-4f56-b323-3e73e0621cac service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:46.976384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48313dd8-16b9-4f56-b323-3e73e0621cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:46.978194 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-48313dd8-16b9-4f56-b323-3e73e0621cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:47.065257 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48313dd8-16b9-4f56-b323-3e73e0621cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:47.066779 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48313dd8-16b9-4f56-b323-3e73e0621cac service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:47.085774 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1408/2781] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:07:47.540180 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6d90e-5a0c-4f51-a53c-572ebfad3939 service placement] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:47.570775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6d90e-5a0c-4f51-a53c-572ebfad3939 service placement] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:47.578078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c98e95-09c8-45ee-b879-8278d947117f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:47.579021 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c98e95-09c8-45ee-b879-8278d947117f service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:47.579371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c98e95-09c8-45ee-b879-8278d947117f service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:47.581328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c98e95-09c8-45ee-b879-8278d947117f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:47.582366 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-54c98e95-09c8-45ee-b879-8278d947117f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:47.596658 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c98e95-09c8-45ee-b879-8278d947117f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:47.598492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c98e95-09c8-45ee-b879-8278d947117f service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:47.611072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1374/2782] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:07:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:07:50.753848 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48313dd8-16b9-4f56-b323-3e73e0621cac service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:50.765885 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48313dd8-16b9-4f56-b323-3e73e0621cac service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:50.771872 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350218f3-eb98-4fd2-8140-268e5b6081f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:50.772509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350218f3-eb98-4fd2-8140-268e5b6081f1 service placement] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:50.772796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350218f3-eb98-4fd2-8140-268e5b6081f1 service placement] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:50.773074 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350218f3-eb98-4fd2-8140-268e5b6081f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:50.774897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-350218f3-eb98-4fd2-8140-268e5b6081f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:50.788483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350218f3-eb98-4fd2-8140-268e5b6081f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:50.790102 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350218f3-eb98-4fd2-8140-268e5b6081f1 service placement] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:50.800823 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1409/2783] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:07:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:07:50.876620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c98e95-09c8-45ee-b879-8278d947117f service nova] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:50.878462 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-54c98e95-09c8-45ee-b879-8278d947117f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:50.892382 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c98e95-09c8-45ee-b879-8278d947117f service nova] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:50.896943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663e680-b0b4-4fd1-8533-9a010c312b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:50.897575 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663e680-b0b4-4fd1-8533-9a010c312b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:50.897855 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663e680-b0b4-4fd1-8533-9a010c312b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:50.898162 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663e680-b0b4-4fd1-8533-9a010c312b2b admin admin] PATH_INFO: `/v3/users/9d1076aed12f4d22a34df10ed0d8e462` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:50.899283 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663e680-b0b4-4fd1-8533-9a010c312b2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d1076aed12f4d22a34df10ed0d8e462)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:50.901235 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663e680-b0b4-4fd1-8533-9a010c312b2b admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:51.092116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4663e680-b0b4-4fd1-8533-9a010c312b2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d1076aed12f4d22a34df10ed0d8e462'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:07:51.092713 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4663e680-b0b4-4fd1-8533-9a010c312b2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d1076aed12f4d22a34df10ed0d8e462'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:07:51.093326 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4663e680-b0b4-4fd1-8533-9a010c312b2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d1076aed12f4d22a34df10ed0d8e462'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:07:51.100179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4663e680-b0b4-4fd1-8533-9a010c312b2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d1076aed12f4d22a34df10ed0d8e462'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:07:51.169905 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-4663e680-b0b4-4fd1-8533-9a010c312b2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d1076aed12f4d22a34df10ed0d8e462'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:07:51.188151 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1375/2784] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:07:50 2022] DELETE /identity/v3/users/9d1076aed12f4d22a34df10ed0d8e462 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:07:51.417142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350218f3-eb98-4fd2-8140-268e5b6081f1 service placement] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:51.430066 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350218f3-eb98-4fd2-8140-268e5b6081f1 service placement] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:51.491916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b186e-510d-4f51-9477-a123921fe9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:51.492582 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b186e-510d-4f51-9477-a123921fe9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:51.492866 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b186e-510d-4f51-9477-a123921fe9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:51.493139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b186e-510d-4f51-9477-a123921fe9b8 admin admin] PATH_INFO: `/v3/projects/19ac17f7efa644819e75576a06bb2499` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:51.494268 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b186e-510d-4f51-9477-a123921fe9b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19ac17f7efa644819e75576a06bb2499)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:51.496844 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b186e-510d-4f51-9477-a123921fe9b8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:51.608393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:51.618154 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-787b186e-510d-4f51-9477-a123921fe9b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19ac17f7efa644819e75576a06bb2499', 'request_id': 'req-787b186e-510d-4f51-9477-a123921fe9b8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:07:51.624390 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-787b186e-510d-4f51-9477-a123921fe9b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19ac17f7efa644819e75576a06bb2499', 'request_id': 'req-787b186e-510d-4f51-9477-a123921fe9b8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:07:51.630428 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-787b186e-510d-4f51-9477-a123921fe9b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19ac17f7efa644819e75576a06bb2499', 'request_id': 'req-787b186e-510d-4f51-9477-a123921fe9b8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:07:51.748299 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-787b186e-510d-4f51-9477-a123921fe9b8 admin admin] The token cache is being invalidate because project 19ac17f7efa644819e75576a06bb2499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:07:51.748762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-787b186e-510d-4f51-9477-a123921fe9b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:07:51.749489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-787b186e-510d-4f51-9477-a123921fe9b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:07:51.751191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1410/2785] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:07:51 2022] DELETE /identity/v3/projects/19ac17f7efa644819e75576a06bb2499 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:07:51.756355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008758c4-d035-4a4b-a1ad-0774d710085c None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:51.756683 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008758c4-d035-4a4b-a1ad-0774d710085c None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:51.757015 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008758c4-d035-4a4b-a1ad-0774d710085c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:51.816567 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-008758c4-d035-4a4b-a1ad-0774d710085c None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:51.817769 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-008758c4-d035-4a4b-a1ad-0774d710085c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:51.879816 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1376/2786] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:07:51.888087 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2988d57-5465-4fd5-b993-fddd695484fd None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:51.888636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2988d57-5465-4fd5-b993-fddd695484fd None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:51.888922 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2988d57-5465-4fd5-b993-fddd695484fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:51.909568 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f2988d57-5465-4fd5-b993-fddd695484fd None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:51.912884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f2988d57-5465-4fd5-b993-fddd695484fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:51.919859 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1411/2787] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:07:51.924110 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008758c4-d035-4a4b-a1ad-0774d710085c None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:51.934345 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008758c4-d035-4a4b-a1ad-0774d710085c None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:51.938932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9426f-12b5-4199-85d6-bd8543d3cef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:51.939526 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9426f-12b5-4199-85d6-bd8543d3cef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:51.939853 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9426f-12b5-4199-85d6-bd8543d3cef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:51.940127 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9426f-12b5-4199-85d6-bd8543d3cef7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:51.940817 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9426f-12b5-4199-85d6-bd8543d3cef7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:51.941123 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9426f-12b5-4199-85d6-bd8543d3cef7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:51.942705 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9426f-12b5-4199-85d6-bd8543d3cef7 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:51.956009 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1377/2788] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 17:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:51.960384 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2988d57-5465-4fd5-b993-fddd695484fd None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:51.971605 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2988d57-5465-4fd5-b993-fddd695484fd None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:51.976577 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73152520-6e8d-4bd1-9835-b50360e006c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:51.977349 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73152520-6e8d-4bd1-9835-b50360e006c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:51.977690 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73152520-6e8d-4bd1-9835-b50360e006c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:51.977964 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73152520-6e8d-4bd1-9835-b50360e006c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:51.978610 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73152520-6e8d-4bd1-9835-b50360e006c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:51.980591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73152520-6e8d-4bd1-9835-b50360e006c0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:52.075697 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1412/2789] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:07:51 2022] POST /identity/v3/projects => generated 367 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:07:52.079899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9426f-12b5-4199-85d6-bd8543d3cef7 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:52.090054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9426f-12b5-4199-85d6-bd8543d3cef7 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:52.094550 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:52.147052 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bcb0a-fae9-4c53-aa45-d85a1e168c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:52.147801 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bcb0a-fae9-4c53-aa45-d85a1e168c32 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:52.148092 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bcb0a-fae9-4c53-aa45-d85a1e168c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:52.148818 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bcb0a-fae9-4c53-aa45-d85a1e168c32 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:52.149535 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bcb0a-fae9-4c53-aa45-d85a1e168c32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:52.151476 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bcb0a-fae9-4c53-aa45-d85a1e168c32 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:52.267898 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1378/2790] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:07:52 2022] POST /identity/v3/users => generated 284 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:07:52.271934 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73152520-6e8d-4bd1-9835-b50360e006c0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:52.282127 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73152520-6e8d-4bd1-9835-b50360e006c0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:52.286701 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814507d-8b30-4b1c-9f96-a8dd03097794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:52.287293 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814507d-8b30-4b1c-9f96-a8dd03097794 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:52.287573 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814507d-8b30-4b1c-9f96-a8dd03097794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:52.287845 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814507d-8b30-4b1c-9f96-a8dd03097794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:52.288564 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814507d-8b30-4b1c-9f96-a8dd03097794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:52.290156 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814507d-8b30-4b1c-9f96-a8dd03097794 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:52.300739 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1413/2791] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:07:52.304882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bcb0a-fae9-4c53-aa45-d85a1e168c32 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:52.330541 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bcb0a-fae9-4c53-aa45-d85a1e168c32 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:52.336507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf09a50-e2c8-414b-aea4-67355f99056b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:52.337124 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf09a50-e2c8-414b-aea4-67355f99056b admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:52.337425 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf09a50-e2c8-414b-aea4-67355f99056b admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:52.337682 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf09a50-e2c8-414b-aea4-67355f99056b admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:52.338330 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf09a50-e2c8-414b-aea4-67355f99056b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:52.339968 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf09a50-e2c8-414b-aea4-67355f99056b admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:52.352599 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1379/2792] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:07:52.357496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814507d-8b30-4b1c-9f96-a8dd03097794 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:52.367573 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814507d-8b30-4b1c-9f96-a8dd03097794 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:52.372864 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e6553-0bdb-46d7-8af1-6e8d99882351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:52.373512 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e6553-0bdb-46d7-8af1-6e8d99882351 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:52.373792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e6553-0bdb-46d7-8af1-6e8d99882351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:52.374086 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e6553-0bdb-46d7-8af1-6e8d99882351 admin admin] PATH_INFO: `/v3/projects/d8895ecbfa934ad5ae0b60c9603818e2/users/c6a61f9490ff4a3c86d8dea4d6a999b9/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:52.389012 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e6553-0bdb-46d7-8af1-6e8d99882351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8895ecbfa934ad5ae0b60c9603818e2, user_id=c6a61f9490ff4a3c86d8dea4d6a999b9, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:52.391724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e6553-0bdb-46d7-8af1-6e8d99882351 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:52.438851 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1414/2793] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:07:52 2022] PUT /identity/v3/projects/d8895ecbfa934ad5ae0b60c9603818e2/users/c6a61f9490ff4a3c86d8dea4d6a999b9/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:07:52.449250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fdc5a-016d-4d2b-a046-05995498a0fc None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:52.449538 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fdc5a-016d-4d2b-a046-05995498a0fc None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:52.449838 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fdc5a-016d-4d2b-a046-05995498a0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:52.476831 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-800fdc5a-016d-4d2b-a046-05995498a0fc None None] MFA Rules not processed for user `c6a61f9490ff4a3c86d8dea4d6a999b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:52.478068 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-800fdc5a-016d-4d2b-a046-05995498a0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:52.540328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1380/2794] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:07:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:07:52.551838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e6553-0bdb-46d7-8af1-6e8d99882351 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:52.553537 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b97e6553-0bdb-46d7-8af1-6e8d99882351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:52.630624 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e6553-0bdb-46d7-8af1-6e8d99882351 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:52.634296 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b393f7d-b1e9-4ad1-8768-e2f7190b2b15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:52.634896 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b393f7d-b1e9-4ad1-8768-e2f7190b2b15 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:52.635174 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b393f7d-b1e9-4ad1-8768-e2f7190b2b15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:52.635471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b393f7d-b1e9-4ad1-8768-e2f7190b2b15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:52.643418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b393f7d-b1e9-4ad1-8768-e2f7190b2b15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:52.644860 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b393f7d-b1e9-4ad1-8768-e2f7190b2b15 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:52.653890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1415/2795] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:07:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:07:55.471467 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fdc5a-016d-4d2b-a046-05995498a0fc None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:55.473120 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-800fdc5a-016d-4d2b-a046-05995498a0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:55.552978 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fdc5a-016d-4d2b-a046-05995498a0fc None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:55.556666 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff0804-7bac-4091-a687-77c77ec0b713 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:55.557878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff0804-7bac-4091-a687-77c77ec0b713 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:55.558161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff0804-7bac-4091-a687-77c77ec0b713 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:55.558436 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff0804-7bac-4091-a687-77c77ec0b713 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:55.560183 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-2aff0804-7bac-4091-a687-77c77ec0b713 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:55.580146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:55.632084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff0804-7bac-4091-a687-77c77ec0b713 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:55.634031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff0804-7bac-4091-a687-77c77ec0b713 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:55.645910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1381/2796] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:07:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:07:57.644065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b393f7d-b1e9-4ad1-8768-e2f7190b2b15 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:57.652909 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b393f7d-b1e9-4ad1-8768-e2f7190b2b15 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:57.657903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8930db4-22e1-4d43-8e24-7e25da5ca430 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:57.657903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8930db4-22e1-4d43-8e24-7e25da5ca430 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:57.657903 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8930db4-22e1-4d43-8e24-7e25da5ca430 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:57.658256 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8930db4-22e1-4d43-8e24-7e25da5ca430 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:57.659764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d8930db4-22e1-4d43-8e24-7e25da5ca430 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:57.689468 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8930db4-22e1-4d43-8e24-7e25da5ca430 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:57.690892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8930db4-22e1-4d43-8e24-7e25da5ca430 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:57.713875 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1416/2797] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:07:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:08:01.347932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff0804-7bac-4091-a687-77c77ec0b713 service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:01.356874 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff0804-7bac-4091-a687-77c77ec0b713 service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:01.360707 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27b785-e62c-4681-aaf2-a0b2109b15a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:01.361305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27b785-e62c-4681-aaf2-a0b2109b15a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:01.361583 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27b785-e62c-4681-aaf2-a0b2109b15a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:01.361856 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27b785-e62c-4681-aaf2-a0b2109b15a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:01.363632 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-4d27b785-e62c-4681-aaf2-a0b2109b15a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:01.427760 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27b785-e62c-4681-aaf2-a0b2109b15a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:01.429182 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27b785-e62c-4681-aaf2-a0b2109b15a0 service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:01.442903 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1382/2798] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:08:02.288093 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8930db4-22e1-4d43-8e24-7e25da5ca430 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:02.298520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8930db4-22e1-4d43-8e24-7e25da5ca430 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:02.303099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c8d37-b7df-4b75-b709-f21211a4b78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:02.303735 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c8d37-b7df-4b75-b709-f21211a4b78a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:02.304003 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c8d37-b7df-4b75-b709-f21211a4b78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:02.304280 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c8d37-b7df-4b75-b709-f21211a4b78a admin admin] PATH_INFO: `/v3/users/c6a61f9490ff4a3c86d8dea4d6a999b9` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:02.305496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c8d37-b7df-4b75-b709-f21211a4b78a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6a61f9490ff4a3c86d8dea4d6a999b9)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:02.307954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c8d37-b7df-4b75-b709-f21211a4b78a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:02.310225 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:02.489446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5c7c8d37-b7df-4b75-b709-f21211a4b78a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6a61f9490ff4a3c86d8dea4d6a999b9'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:02.490496 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5c7c8d37-b7df-4b75-b709-f21211a4b78a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6a61f9490ff4a3c86d8dea4d6a999b9'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:02.490743 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5c7c8d37-b7df-4b75-b709-f21211a4b78a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6a61f9490ff4a3c86d8dea4d6a999b9'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:02.559004 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5c7c8d37-b7df-4b75-b709-f21211a4b78a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6a61f9490ff4a3c86d8dea4d6a999b9'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:02.569776 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5c7c8d37-b7df-4b75-b709-f21211a4b78a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6a61f9490ff4a3c86d8dea4d6a999b9'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:02.589251 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1417/2799] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:08:02 2022] DELETE /identity/v3/users/c6a61f9490ff4a3c86d8dea4d6a999b9 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:02.593692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8b118-a4b9-48e8-a5f2-aee78ecf2789 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:02.594170 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8b118-a4b9-48e8-a5f2-aee78ecf2789 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:02.594263 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8b118-a4b9-48e8-a5f2-aee78ecf2789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:02.612959 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-61f8b118-a4b9-48e8-a5f2-aee78ecf2789 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:02.614096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-61f8b118-a4b9-48e8-a5f2-aee78ecf2789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:02.682910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1383/2800] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:02.691206 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c8d37-b7df-4b75-b709-f21211a4b78a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:02.693252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5c7c8d37-b7df-4b75-b709-f21211a4b78a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:02.751218 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c8d37-b7df-4b75-b709-f21211a4b78a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:02.755049 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086dc41-98cc-4b90-91ce-b120422ea7e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:02.755659 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086dc41-98cc-4b90-91ce-b120422ea7e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:02.755940 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086dc41-98cc-4b90-91ce-b120422ea7e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:02.756214 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086dc41-98cc-4b90-91ce-b120422ea7e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:02.765627 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086dc41-98cc-4b90-91ce-b120422ea7e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:02.767059 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086dc41-98cc-4b90-91ce-b120422ea7e4 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:02.777132 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1418/2801] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:08:03.367448 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8b118-a4b9-48e8-a5f2-aee78ecf2789 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:03.369049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-61f8b118-a4b9-48e8-a5f2-aee78ecf2789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:03.398551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8b118-a4b9-48e8-a5f2-aee78ecf2789 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:03.404316 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b846c5-309b-47f6-bcd2-ccd22e5335d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:03.405190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b846c5-309b-47f6-bcd2-ccd22e5335d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:03.405776 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b846c5-309b-47f6-bcd2-ccd22e5335d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:03.405776 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b846c5-309b-47f6-bcd2-ccd22e5335d3 admin admin] PATH_INFO: `/v3/projects/d8895ecbfa934ad5ae0b60c9603818e2` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:03.407032 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b846c5-309b-47f6-bcd2-ccd22e5335d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8895ecbfa934ad5ae0b60c9603818e2)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:03.410044 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b846c5-309b-47f6-bcd2-ccd22e5335d3 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:03.506780 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:03.515309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-88b846c5-309b-47f6-bcd2-ccd22e5335d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8895ecbfa934ad5ae0b60c9603818e2', 'request_id': 'req-88b846c5-309b-47f6-bcd2-ccd22e5335d3'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:03.521875 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-88b846c5-309b-47f6-bcd2-ccd22e5335d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8895ecbfa934ad5ae0b60c9603818e2', 'request_id': 'req-88b846c5-309b-47f6-bcd2-ccd22e5335d3'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:03.527643 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-88b846c5-309b-47f6-bcd2-ccd22e5335d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8895ecbfa934ad5ae0b60c9603818e2', 'request_id': 'req-88b846c5-309b-47f6-bcd2-ccd22e5335d3'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:03.592292 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-88b846c5-309b-47f6-bcd2-ccd22e5335d3 admin admin] The token cache is being invalidate because project d8895ecbfa934ad5ae0b60c9603818e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:08:03.592612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-88b846c5-309b-47f6-bcd2-ccd22e5335d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:03.593256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-88b846c5-309b-47f6-bcd2-ccd22e5335d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:03.594785 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1384/2802] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:08:03 2022] DELETE /identity/v3/projects/d8895ecbfa934ad5ae0b60c9603818e2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:03.599593 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fdeba8-2319-497c-84a5-453e75b35809 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:03.600013 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fdeba8-2319-497c-84a5-453e75b35809 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:03.600141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fdeba8-2319-497c-84a5-453e75b35809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:03.621099 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c2fdeba8-2319-497c-84a5-453e75b35809 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:03.622634 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c2fdeba8-2319-497c-84a5-453e75b35809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:03.688431 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1419/2803] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:03.697289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b401a6-a262-4500-ad81-04adc0d9d286 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:03.697771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b401a6-a262-4500-ad81-04adc0d9d286 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:03.698120 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b401a6-a262-4500-ad81-04adc0d9d286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:03.718121 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-d5b401a6-a262-4500-ad81-04adc0d9d286 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:03.719325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-d5b401a6-a262-4500-ad81-04adc0d9d286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:03.727291 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1385/2804] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:03.732197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fdeba8-2319-497c-84a5-453e75b35809 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:03.741855 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fdeba8-2319-497c-84a5-453e75b35809 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:03.746369 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c343d-29ca-47a7-81e8-d89c16491a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:03.746905 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c343d-29ca-47a7-81e8-d89c16491a91 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:03.747209 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c343d-29ca-47a7-81e8-d89c16491a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:03.747477 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c343d-29ca-47a7-81e8-d89c16491a91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:03.748112 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c343d-29ca-47a7-81e8-d89c16491a91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:03.748421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c343d-29ca-47a7-81e8-d89c16491a91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:03.750935 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c343d-29ca-47a7-81e8-d89c16491a91 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:03.762247 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1420/2805] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 17:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:03.766720 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b401a6-a262-4500-ad81-04adc0d9d286 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:03.776105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b401a6-a262-4500-ad81-04adc0d9d286 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:03.783737 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f95b11-9dc3-4f1e-99b3-de7da2f1e95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:03.784459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f95b11-9dc3-4f1e-99b3-de7da2f1e95d admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:03.784807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f95b11-9dc3-4f1e-99b3-de7da2f1e95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:03.785636 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f95b11-9dc3-4f1e-99b3-de7da2f1e95d admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:03.786350 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f95b11-9dc3-4f1e-99b3-de7da2f1e95d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:03.788457 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f95b11-9dc3-4f1e-99b3-de7da2f1e95d admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:03.904157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1386/2806] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:08:03 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:08:03.908538 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c343d-29ca-47a7-81e8-d89c16491a91 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:03.919102 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c343d-29ca-47a7-81e8-d89c16491a91 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:03.928128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:03.978661 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae3317-c702-4c0c-a202-1cc0b99abf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:03.979833 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae3317-c702-4c0c-a202-1cc0b99abf18 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:03.980647 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae3317-c702-4c0c-a202-1cc0b99abf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:03.980647 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae3317-c702-4c0c-a202-1cc0b99abf18 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:03.981043 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae3317-c702-4c0c-a202-1cc0b99abf18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:03.982964 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae3317-c702-4c0c-a202-1cc0b99abf18 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:04.084884 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1421/2807] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:08:03 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:08:04.089737 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f95b11-9dc3-4f1e-99b3-de7da2f1e95d admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:04.100076 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f95b11-9dc3-4f1e-99b3-de7da2f1e95d admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:04.105342 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48346a32-d38d-428c-b369-d056dcd98368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:04.105891 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48346a32-d38d-428c-b369-d056dcd98368 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:04.111180 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48346a32-d38d-428c-b369-d056dcd98368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:04.111180 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48346a32-d38d-428c-b369-d056dcd98368 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:04.111180 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48346a32-d38d-428c-b369-d056dcd98368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:04.111180 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48346a32-d38d-428c-b369-d056dcd98368 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:04.119215 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1387/2808] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:04.123133 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae3317-c702-4c0c-a202-1cc0b99abf18 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:04.139921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae3317-c702-4c0c-a202-1cc0b99abf18 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:04.139921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee02661-7836-484a-9d7f-531e2c905934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:04.139921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee02661-7836-484a-9d7f-531e2c905934 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:04.139921 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee02661-7836-484a-9d7f-531e2c905934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:04.140434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee02661-7836-484a-9d7f-531e2c905934 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:04.140434 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee02661-7836-484a-9d7f-531e2c905934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:04.141813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee02661-7836-484a-9d7f-531e2c905934 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:04.153048 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1422/2809] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:04.157313 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48346a32-d38d-428c-b369-d056dcd98368 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:04.168036 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48346a32-d38d-428c-b369-d056dcd98368 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:04.172657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497773d9-933a-45a8-9503-5ac307a7278b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:04.173306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497773d9-933a-45a8-9503-5ac307a7278b admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:04.173587 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497773d9-933a-45a8-9503-5ac307a7278b admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:04.173860 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497773d9-933a-45a8-9503-5ac307a7278b admin admin] PATH_INFO: `/v3/projects/07aa12bfd07e487e83c90306d83164f1/users/bed4f57d853d460b8ed1ad622040538b/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:04.188800 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497773d9-933a-45a8-9503-5ac307a7278b admin admin] RBAC: Authorizing `identity:create_grant(project_id=07aa12bfd07e487e83c90306d83164f1, user_id=bed4f57d853d460b8ed1ad622040538b, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:04.191479 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497773d9-933a-45a8-9503-5ac307a7278b admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:04.310521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1388/2810] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:08:04 2022] PUT /identity/v3/projects/07aa12bfd07e487e83c90306d83164f1/users/bed4f57d853d460b8ed1ad622040538b/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:04.321169 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3af5f-6342-41e7-8751-1cc6e4e857e5 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:04.323479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3af5f-6342-41e7-8751-1cc6e4e857e5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:04.323479 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3af5f-6342-41e7-8751-1cc6e4e857e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:04.374899 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ded3af5f-6342-41e7-8751-1cc6e4e857e5 None None] MFA Rules not processed for user `bed4f57d853d460b8ed1ad622040538b`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:04.376290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ded3af5f-6342-41e7-8751-1cc6e4e857e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:04.469990 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1423/2811] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:04.481462 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497773d9-933a-45a8-9503-5ac307a7278b admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:04.483397 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-497773d9-933a-45a8-9503-5ac307a7278b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:04.591307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497773d9-933a-45a8-9503-5ac307a7278b admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:04.597120 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa6ba2-b1ec-4f38-bcda-318254991f8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e333fca7790244f6a4ebceabb55faba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:04.597723 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa6ba2-b1ec-4f38-bcda-318254991f8f service nova] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:04.598004 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa6ba2-b1ec-4f38-bcda-318254991f8f service nova] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:04.598275 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa6ba2-b1ec-4f38-bcda-318254991f8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:04.606610 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa6ba2-b1ec-4f38-bcda-318254991f8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:04.608226 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa6ba2-b1ec-4f38-bcda-318254991f8f service nova] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:04.617442 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1389/2812] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:08:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:08:04.867567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3af5f-6342-41e7-8751-1cc6e4e857e5 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:04.942539 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3af5f-6342-41e7-8751-1cc6e4e857e5 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:04.947060 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a35846-f18d-4cc8-9ac0-d3ea4647e261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:04.948230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a35846-f18d-4cc8-9ac0-d3ea4647e261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:04.948541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a35846-f18d-4cc8-9ac0-d3ea4647e261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:04.948955 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a35846-f18d-4cc8-9ac0-d3ea4647e261 admin admin] PATH_INFO: `/v3/users/bed4f57d853d460b8ed1ad622040538b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:04.950080 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a35846-f18d-4cc8-9ac0-d3ea4647e261 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bed4f57d853d460b8ed1ad622040538b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:04.951980 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a35846-f18d-4cc8-9ac0-d3ea4647e261 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:04.973393 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:05.097904 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-79a35846-f18d-4cc8-9ac0-d3ea4647e261 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bed4f57d853d460b8ed1ad622040538b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:05.098566 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-79a35846-f18d-4cc8-9ac0-d3ea4647e261 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bed4f57d853d460b8ed1ad622040538b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:05.099179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-79a35846-f18d-4cc8-9ac0-d3ea4647e261 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bed4f57d853d460b8ed1ad622040538b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:05.159149 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-79a35846-f18d-4cc8-9ac0-d3ea4647e261 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bed4f57d853d460b8ed1ad622040538b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:05.167167 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-79a35846-f18d-4cc8-9ac0-d3ea4647e261 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bed4f57d853d460b8ed1ad622040538b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:05.185590 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1424/2813] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:08:04 2022] DELETE /identity/v3/users/bed4f57d853d460b8ed1ad622040538b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:05.190197 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180d2bb-981f-4e23-a32d-c14f33bc0eed None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.190482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180d2bb-981f-4e23-a32d-c14f33bc0eed None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.190775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180d2bb-981f-4e23-a32d-c14f33bc0eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.210348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-6180d2bb-981f-4e23-a32d-c14f33bc0eed None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:05.211565 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6180d2bb-981f-4e23-a32d-c14f33bc0eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:05.249589 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:05.281278 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1390/2814] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:08:05.288252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a35846-f18d-4cc8-9ac0-d3ea4647e261 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:05.289993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-79a35846-f18d-4cc8-9ac0-d3ea4647e261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:05.350379 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a35846-f18d-4cc8-9ac0-d3ea4647e261 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:05.354080 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bc94f-10cf-4944-8453-1eebe4fc9d6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:05.354695 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bc94f-10cf-4944-8453-1eebe4fc9d6d service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.354976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bc94f-10cf-4944-8453-1eebe4fc9d6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.355248 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bc94f-10cf-4944-8453-1eebe4fc9d6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.364695 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bc94f-10cf-4944-8453-1eebe4fc9d6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:05.366126 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bc94f-10cf-4944-8453-1eebe4fc9d6d service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:05.376440 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1425/2815] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:08:05.794897 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180d2bb-981f-4e23-a32d-c14f33bc0eed None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:05.797096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-6180d2bb-981f-4e23-a32d-c14f33bc0eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:05.811985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180d2bb-981f-4e23-a32d-c14f33bc0eed None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:05.817298 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3267b-1271-419e-9737-bc3ea03557b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:05.817967 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3267b-1271-419e-9737-bc3ea03557b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.818220 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3267b-1271-419e-9737-bc3ea03557b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.818488 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3267b-1271-419e-9737-bc3ea03557b0 admin admin] PATH_INFO: `/v3/projects/07aa12bfd07e487e83c90306d83164f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.820615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3267b-1271-419e-9737-bc3ea03557b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07aa12bfd07e487e83c90306d83164f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:05.823438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3267b-1271-419e-9737-bc3ea03557b0 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:05.921460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-38b3267b-1271-419e-9737-bc3ea03557b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07aa12bfd07e487e83c90306d83164f1', 'request_id': 'req-38b3267b-1271-419e-9737-bc3ea03557b0'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:05.928985 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-38b3267b-1271-419e-9737-bc3ea03557b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07aa12bfd07e487e83c90306d83164f1', 'request_id': 'req-38b3267b-1271-419e-9737-bc3ea03557b0'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:05.936413 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-38b3267b-1271-419e-9737-bc3ea03557b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07aa12bfd07e487e83c90306d83164f1', 'request_id': 'req-38b3267b-1271-419e-9737-bc3ea03557b0'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:06.015961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-38b3267b-1271-419e-9737-bc3ea03557b0 admin admin] The token cache is being invalidate because project 07aa12bfd07e487e83c90306d83164f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:08:06.016256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-38b3267b-1271-419e-9737-bc3ea03557b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:06.016938 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-38b3267b-1271-419e-9737-bc3ea03557b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:06.018619 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1391/2816] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:08:05 2022] DELETE /identity/v3/projects/07aa12bfd07e487e83c90306d83164f1 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:06.024205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13d3db-a4d8-435e-a072-7aaaeda86643 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.024205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13d3db-a4d8-435e-a072-7aaaeda86643 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.024205 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13d3db-a4d8-435e-a072-7aaaeda86643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.045419 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-dd13d3db-a4d8-435e-a072-7aaaeda86643 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:06.046615 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dd13d3db-a4d8-435e-a072-7aaaeda86643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:06.106331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1426/2817] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:06.114251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997dc6e-1aa7-4b9c-ad09-4471102fed42 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.114615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997dc6e-1aa7-4b9c-ad09-4471102fed42 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.114893 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997dc6e-1aa7-4b9c-ad09-4471102fed42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.147607 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-1997dc6e-1aa7-4b9c-ad09-4471102fed42 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:06.148865 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-1997dc6e-1aa7-4b9c-ad09-4471102fed42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:06.158072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1392/2818] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:06.167319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13d3db-a4d8-435e-a072-7aaaeda86643 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.172528 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13d3db-a4d8-435e-a072-7aaaeda86643 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.177285 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a99d4-32f4-492e-80be-4f0415856224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.177893 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a99d4-32f4-492e-80be-4f0415856224 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.178178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a99d4-32f4-492e-80be-4f0415856224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.178456 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a99d4-32f4-492e-80be-4f0415856224 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.179107 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a99d4-32f4-492e-80be-4f0415856224 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:06.179506 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a99d4-32f4-492e-80be-4f0415856224 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.181183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a99d4-32f4-492e-80be-4f0415856224 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.202164 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1427/2819] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 17:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:06.206640 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997dc6e-1aa7-4b9c-ad09-4471102fed42 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.216810 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997dc6e-1aa7-4b9c-ad09-4471102fed42 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.221955 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fb074-89e8-4e87-8a9a-2999977835bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.222577 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fb074-89e8-4e87-8a9a-2999977835bc admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.222858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fb074-89e8-4e87-8a9a-2999977835bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.223130 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fb074-89e8-4e87-8a9a-2999977835bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.223770 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fb074-89e8-4e87-8a9a-2999977835bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.226011 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fb074-89e8-4e87-8a9a-2999977835bc admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.313010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1393/2820] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:08:06 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:08:06.317554 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a99d4-32f4-492e-80be-4f0415856224 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.331886 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a99d4-32f4-492e-80be-4f0415856224 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.362972 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:06.398576 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73220e0-e63b-498e-9eb8-b86c98f025b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.399187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73220e0-e63b-498e-9eb8-b86c98f025b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.400178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73220e0-e63b-498e-9eb8-b86c98f025b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.400178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73220e0-e63b-498e-9eb8-b86c98f025b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.401024 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73220e0-e63b-498e-9eb8-b86c98f025b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.402862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73220e0-e63b-498e-9eb8-b86c98f025b0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.489635 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1428/2821] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:08:06 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:06.493743 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fb074-89e8-4e87-8a9a-2999977835bc admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.504205 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fb074-89e8-4e87-8a9a-2999977835bc admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.508776 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd4937-7227-4b86-9cbe-b561d868a298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.509366 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd4937-7227-4b86-9cbe-b561d868a298 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.510132 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd4937-7227-4b86-9cbe-b561d868a298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.510409 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd4937-7227-4b86-9cbe-b561d868a298 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.511041 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd4937-7227-4b86-9cbe-b561d868a298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.512637 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd4937-7227-4b86-9cbe-b561d868a298 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.522611 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1394/2822] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:06.527233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73220e0-e63b-498e-9eb8-b86c98f025b0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.537442 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73220e0-e63b-498e-9eb8-b86c98f025b0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.542331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616e9f6-235b-4887-a9a0-ebcf6881f0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.542950 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616e9f6-235b-4887-a9a0-ebcf6881f0dd admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.543611 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616e9f6-235b-4887-a9a0-ebcf6881f0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.543611 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616e9f6-235b-4887-a9a0-ebcf6881f0dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.544106 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616e9f6-235b-4887-a9a0-ebcf6881f0dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.545738 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616e9f6-235b-4887-a9a0-ebcf6881f0dd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.556549 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1429/2823] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:06.563850 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd4937-7227-4b86-9cbe-b561d868a298 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.574370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd4937-7227-4b86-9cbe-b561d868a298 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.579036 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbff74e-9834-4a2c-a921-f0d5211fd25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.579680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbff74e-9834-4a2c-a921-f0d5211fd25c admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.579960 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbff74e-9834-4a2c-a921-f0d5211fd25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.580233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbff74e-9834-4a2c-a921-f0d5211fd25c admin admin] PATH_INFO: `/v3/projects/9d07ad04f5a54f92a3b9ccc5fee960c6/users/bded7119a7974da9885a23cbbc787f49/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.595189 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbff74e-9834-4a2c-a921-f0d5211fd25c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d07ad04f5a54f92a3b9ccc5fee960c6, user_id=bded7119a7974da9885a23cbbc787f49, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.597917 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbff74e-9834-4a2c-a921-f0d5211fd25c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.696459 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1395/2824] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:08:06 2022] PUT /identity/v3/projects/9d07ad04f5a54f92a3b9ccc5fee960c6/users/bded7119a7974da9885a23cbbc787f49/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:06.708520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85fc3a-9762-4337-8854-9565cc2d8cf8 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.708900 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85fc3a-9762-4337-8854-9565cc2d8cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.709264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85fc3a-9762-4337-8854-9565cc2d8cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.746051 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3c85fc3a-9762-4337-8854-9565cc2d8cf8 None None] MFA Rules not processed for user `bded7119a7974da9885a23cbbc787f49`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:06.747398 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3c85fc3a-9762-4337-8854-9565cc2d8cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:06.821525 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1430/2825] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:06.826096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbff74e-9834-4a2c-a921-f0d5211fd25c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.895956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:06.899910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbff74e-9834-4a2c-a921-f0d5211fd25c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.905885 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb1deb-bae9-461c-8598-d0e2a25d6729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.906502 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb1deb-bae9-461c-8598-d0e2a25d6729 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.914665 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb1deb-bae9-461c-8598-d0e2a25d6729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.914665 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb1deb-bae9-461c-8598-d0e2a25d6729 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.914665 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb1deb-bae9-461c-8598-d0e2a25d6729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.914665 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb1deb-bae9-461c-8598-d0e2a25d6729 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.032833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1396/2826] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:08:06 2022] POST /identity/v3/projects => generated 375 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:07.037304 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85fc3a-9762-4337-8854-9565cc2d8cf8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.049039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85fc3a-9762-4337-8854-9565cc2d8cf8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.124910 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed76bb5-b5f0-4132-8d42-906fa434f2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.125508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed76bb5-b5f0-4132-8d42-906fa434f2c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.125798 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed76bb5-b5f0-4132-8d42-906fa434f2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.126072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed76bb5-b5f0-4132-8d42-906fa434f2c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.126724 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed76bb5-b5f0-4132-8d42-906fa434f2c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.129557 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed76bb5-b5f0-4132-8d42-906fa434f2c6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.223525 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1431/2827] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:08:07 2022] POST /identity/v3/users => generated 294 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:07.228560 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb1deb-bae9-461c-8598-d0e2a25d6729 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.239337 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb1deb-bae9-461c-8598-d0e2a25d6729 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.244054 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd9409-c9d7-452c-9a0f-af5cfdf19b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.244688 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd9409-c9d7-452c-9a0f-af5cfdf19b45 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.244974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd9409-c9d7-452c-9a0f-af5cfdf19b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.245250 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd9409-c9d7-452c-9a0f-af5cfdf19b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.245916 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd9409-c9d7-452c-9a0f-af5cfdf19b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.247521 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd9409-c9d7-452c-9a0f-af5cfdf19b45 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.258986 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1397/2828] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:07.263969 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed76bb5-b5f0-4132-8d42-906fa434f2c6 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.274508 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed76bb5-b5f0-4132-8d42-906fa434f2c6 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.281939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c10202-1c33-4813-a528-c38160954be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.282616 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c10202-1c33-4813-a528-c38160954be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.282901 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c10202-1c33-4813-a528-c38160954be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.283256 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c10202-1c33-4813-a528-c38160954be4 admin admin] PATH_INFO: `/v3/projects/519ce96682774cbf871033507b0d3fce/users/c92b5a5d73594fd595411d45861eeceb/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.301544 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c10202-1c33-4813-a528-c38160954be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=519ce96682774cbf871033507b0d3fce, user_id=c92b5a5d73594fd595411d45861eeceb, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.304252 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c10202-1c33-4813-a528-c38160954be4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.370590 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1432/2829] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:08:07 2022] PUT /identity/v3/projects/519ce96682774cbf871033507b0d3fce/users/c92b5a5d73594fd595411d45861eeceb/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:07.374617 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd9409-c9d7-452c-9a0f-af5cfdf19b45 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.385625 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd9409-c9d7-452c-9a0f-af5cfdf19b45 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.445805 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28e5e2-4fd6-476b-bde4-7ffa06e78356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.446421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28e5e2-4fd6-476b-bde4-7ffa06e78356 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.446706 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28e5e2-4fd6-476b-bde4-7ffa06e78356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.447506 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28e5e2-4fd6-476b-bde4-7ffa06e78356 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.448191 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28e5e2-4fd6-476b-bde4-7ffa06e78356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.449822 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28e5e2-4fd6-476b-bde4-7ffa06e78356 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.459812 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1398/2830] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:07.463792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c10202-1c33-4813-a528-c38160954be4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.469470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:07.474757 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c10202-1c33-4813-a528-c38160954be4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.480084 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7756679-f65a-4ede-9e02-0b4b986b489b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.480750 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7756679-f65a-4ede-9e02-0b4b986b489b admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.481032 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7756679-f65a-4ede-9e02-0b4b986b489b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.481308 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7756679-f65a-4ede-9e02-0b4b986b489b admin admin] PATH_INFO: `/v3/projects/519ce96682774cbf871033507b0d3fce/users/c92b5a5d73594fd595411d45861eeceb/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.483236 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7756679-f65a-4ede-9e02-0b4b986b489b admin admin] RBAC: Authorizing `identity:create_grant(project_id=519ce96682774cbf871033507b0d3fce, user_id=c92b5a5d73594fd595411d45861eeceb, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.485936 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7756679-f65a-4ede-9e02-0b4b986b489b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.503108 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1433/2831] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:08:07 2022] PUT /identity/v3/projects/519ce96682774cbf871033507b0d3fce/users/c92b5a5d73594fd595411d45861eeceb/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:07.514097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f0828-ada5-4792-acb4-c916163da988 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.514391 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f0828-ada5-4792-acb4-c916163da988 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.514672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f0828-ada5-4792-acb4-c916163da988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.545712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-3e0f0828-ada5-4792-acb4-c916163da988 None None] MFA Rules not processed for user `c92b5a5d73594fd595411d45861eeceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:07.546936 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3e0f0828-ada5-4792-acb4-c916163da988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:07.629571 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1399/2832] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:07.637517 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7756679-f65a-4ede-9e02-0b4b986b489b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.642257 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b7756679-f65a-4ede-9e02-0b4b986b489b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:07.706498 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7756679-f65a-4ede-9e02-0b4b986b489b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.710981 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3526b-d8f6-467d-8aca-9942e51576ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.711610 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3526b-d8f6-467d-8aca-9942e51576ae service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.711886 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3526b-d8f6-467d-8aca-9942e51576ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.712160 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3526b-d8f6-467d-8aca-9942e51576ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.778641 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3526b-d8f6-467d-8aca-9942e51576ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.780143 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3526b-d8f6-467d-8aca-9942e51576ae service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.789000 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1434/2833] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:08:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:08:09.028698 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f0828-ada5-4792-acb4-c916163da988 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:09.030276 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3e0f0828-ada5-4792-acb4-c916163da988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:09.083837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:09.095880 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f0828-ada5-4792-acb4-c916163da988 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:09.100347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec6b64-065a-4bd6-bf94-1cfe73b3354c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:09.102038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec6b64-065a-4bd6-bf94-1cfe73b3354c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:09.102038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec6b64-065a-4bd6-bf94-1cfe73b3354c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:09.102038 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec6b64-065a-4bd6-bf94-1cfe73b3354c admin admin] PATH_INFO: `/v3/users/701be5a3cff34016ab22b6dc4d23d939` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:09.114573 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec6b64-065a-4bd6-bf94-1cfe73b3354c admin admin] RBAC: Authorizing `identity:delete_user(user_id=701be5a3cff34016ab22b6dc4d23d939)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:09.116605 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec6b64-065a-4bd6-bf94-1cfe73b3354c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:09.343003 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-eaec6b64-065a-4bd6-bf94-1cfe73b3354c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '701be5a3cff34016ab22b6dc4d23d939'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:09.343584 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-eaec6b64-065a-4bd6-bf94-1cfe73b3354c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '701be5a3cff34016ab22b6dc4d23d939'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:09.344190 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-eaec6b64-065a-4bd6-bf94-1cfe73b3354c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '701be5a3cff34016ab22b6dc4d23d939'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:09.351097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-eaec6b64-065a-4bd6-bf94-1cfe73b3354c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '701be5a3cff34016ab22b6dc4d23d939'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:09.449221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-eaec6b64-065a-4bd6-bf94-1cfe73b3354c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '701be5a3cff34016ab22b6dc4d23d939'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:09.466657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1400/2834] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:08:09 2022] DELETE /identity/v3/users/701be5a3cff34016ab22b6dc4d23d939 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:09.732295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3526b-d8f6-467d-8aca-9942e51576ae service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:09.743959 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3526b-d8f6-467d-8aca-9942e51576ae service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:09.805386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:09.806018 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:09.806295 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:09.807181 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758 admin admin] PATH_INFO: `/v3/projects/805d6de24ff4415c89cba2d318ffd2f4` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:09.818758 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=805d6de24ff4415c89cba2d318ffd2f4)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:09.820957 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:09.967151 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:09.975650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '805d6de24ff4415c89cba2d318ffd2f4', 'request_id': 'req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:09.982671 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '805d6de24ff4415c89cba2d318ffd2f4', 'request_id': 'req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:09.988139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '805d6de24ff4415c89cba2d318ffd2f4', 'request_id': 'req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:10.124141 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758 admin admin] The token cache is being invalidate because project 805d6de24ff4415c89cba2d318ffd2f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:08:10.124455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:10.125101 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:10.127825 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1435/2835] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:08:09 2022] DELETE /identity/v3/projects/805d6de24ff4415c89cba2d318ffd2f4 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:10.130830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec6b64-065a-4bd6-bf94-1cfe73b3354c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:10.132201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-eaec6b64-065a-4bd6-bf94-1cfe73b3354c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:10.147287 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec6b64-065a-4bd6-bf94-1cfe73b3354c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:10.219450 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650762d1-3105-4b15-a2f7-7f02ca97054e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:10.220078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650762d1-3105-4b15-a2f7-7f02ca97054e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:10.220355 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650762d1-3105-4b15-a2f7-7f02ca97054e admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:10.220656 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650762d1-3105-4b15-a2f7-7f02ca97054e admin admin] PATH_INFO: `/v3/users/cb6f2c69360d4b8a9b242415035b335c` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:10.221775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650762d1-3105-4b15-a2f7-7f02ca97054e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb6f2c69360d4b8a9b242415035b335c)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:10.223677 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650762d1-3105-4b15-a2f7-7f02ca97054e admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:10.429477 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-650762d1-3105-4b15-a2f7-7f02ca97054e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb6f2c69360d4b8a9b242415035b335c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:10.430087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-650762d1-3105-4b15-a2f7-7f02ca97054e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb6f2c69360d4b8a9b242415035b335c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:10.430684 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-650762d1-3105-4b15-a2f7-7f02ca97054e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb6f2c69360d4b8a9b242415035b335c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:10.438281 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-650762d1-3105-4b15-a2f7-7f02ca97054e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb6f2c69360d4b8a9b242415035b335c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:10.506536 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-650762d1-3105-4b15-a2f7-7f02ca97054e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb6f2c69360d4b8a9b242415035b335c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:10.507703 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:10.523987 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1401/2836] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:08:10 2022] DELETE /identity/v3/users/cb6f2c69360d4b8a9b242415035b335c => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:10.742771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:10.744230 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:10.757707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a97ae5-a4b2-4e48-aa68-ef8798e64758 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:10.821952 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d77558-c6ac-4246-bcd5-599dd99e9c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:10.822617 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d77558-c6ac-4246-bcd5-599dd99e9c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:10.822898 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d77558-c6ac-4246-bcd5-599dd99e9c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:10.823200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d77558-c6ac-4246-bcd5-599dd99e9c10 admin admin] PATH_INFO: `/v3/projects/a517d3953b9f4ef4ae73e2fc3d83ec4f` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:10.824319 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d77558-c6ac-4246-bcd5-599dd99e9c10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a517d3953b9f4ef4ae73e2fc3d83ec4f)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:10.826323 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d77558-c6ac-4246-bcd5-599dd99e9c10 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:10.948238 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-40d77558-c6ac-4246-bcd5-599dd99e9c10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a517d3953b9f4ef4ae73e2fc3d83ec4f', 'request_id': 'req-40d77558-c6ac-4246-bcd5-599dd99e9c10'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:10.954405 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-40d77558-c6ac-4246-bcd5-599dd99e9c10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a517d3953b9f4ef4ae73e2fc3d83ec4f', 'request_id': 'req-40d77558-c6ac-4246-bcd5-599dd99e9c10'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:10.959929 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-40d77558-c6ac-4246-bcd5-599dd99e9c10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a517d3953b9f4ef4ae73e2fc3d83ec4f', 'request_id': 'req-40d77558-c6ac-4246-bcd5-599dd99e9c10'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:11.031719 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-40d77558-c6ac-4246-bcd5-599dd99e9c10 admin admin] The token cache is being invalidate because project a517d3953b9f4ef4ae73e2fc3d83ec4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:08:11.032005 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-40d77558-c6ac-4246-bcd5-599dd99e9c10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:11.032666 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-40d77558-c6ac-4246-bcd5-599dd99e9c10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:11.034163 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1436/2837] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:08:10 2022] DELETE /identity/v3/projects/a517d3953b9f4ef4ae73e2fc3d83ec4f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:15.335525 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650762d1-3105-4b15-a2f7-7f02ca97054e admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:15.336998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-650762d1-3105-4b15-a2f7-7f02ca97054e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:15.394284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650762d1-3105-4b15-a2f7-7f02ca97054e admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:15.441592 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319d696-26a7-47f0-a036-2929caa0f128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:15.447649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319d696-26a7-47f0-a036-2929caa0f128 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:15.447649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319d696-26a7-47f0-a036-2929caa0f128 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:15.447649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319d696-26a7-47f0-a036-2929caa0f128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:15.447649 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a319d696-26a7-47f0-a036-2929caa0f128 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:15.507400 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319d696-26a7-47f0-a036-2929caa0f128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:15.508881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319d696-26a7-47f0-a036-2929caa0f128 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:15.518878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1402/2838] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:08:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:08:21.028656 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:25.514590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=77,keystone:DELETE=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:45.947416 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d77558-c6ac-4246-bcd5-599dd99e9c10 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:45.948967 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-40d77558-c6ac-4246-bcd5-599dd99e9c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:46.014927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d77558-c6ac-4246-bcd5-599dd99e9c10 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:46.020069 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5794ad-f693-4a4b-aa1c-60df1ad46525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:46.020732 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5794ad-f693-4a4b-aa1c-60df1ad46525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:46.021018 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5794ad-f693-4a4b-aa1c-60df1ad46525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:46.021316 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5794ad-f693-4a4b-aa1c-60df1ad46525 admin admin] PATH_INFO: `/v3/users/bded7119a7974da9885a23cbbc787f49` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:46.022477 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5794ad-f693-4a4b-aa1c-60df1ad46525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bded7119a7974da9885a23cbbc787f49)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:46.024371 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5794ad-f693-4a4b-aa1c-60df1ad46525 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:46.204523 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be5794ad-f693-4a4b-aa1c-60df1ad46525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bded7119a7974da9885a23cbbc787f49'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:46.205213 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be5794ad-f693-4a4b-aa1c-60df1ad46525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bded7119a7974da9885a23cbbc787f49'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:46.205828 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be5794ad-f693-4a4b-aa1c-60df1ad46525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bded7119a7974da9885a23cbbc787f49'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:46.276054 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be5794ad-f693-4a4b-aa1c-60df1ad46525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bded7119a7974da9885a23cbbc787f49'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:46.283016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be5794ad-f693-4a4b-aa1c-60df1ad46525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bded7119a7974da9885a23cbbc787f49'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:46.300974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1437/2839] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:08:45 2022] DELETE /identity/v3/users/bded7119a7974da9885a23cbbc787f49 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:46.305428 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373f733-2d65-4e4f-9af4-862f94ac2842 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:46.308751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373f733-2d65-4e4f-9af4-862f94ac2842 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:46.308751 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373f733-2d65-4e4f-9af4-862f94ac2842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:46.325344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-3373f733-2d65-4e4f-9af4-862f94ac2842 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:46.326534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3373f733-2d65-4e4f-9af4-862f94ac2842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:46.385297 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1403/2840] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:46.397191 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5794ad-f693-4a4b-aa1c-60df1ad46525 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:46.398622 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-be5794ad-f693-4a4b-aa1c-60df1ad46525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:46.454493 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5794ad-f693-4a4b-aa1c-60df1ad46525 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:46.458836 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25623e4a-398a-4efa-897a-fdf4b1cf1204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:46.459380 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25623e4a-398a-4efa-897a-fdf4b1cf1204 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:46.459657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25623e4a-398a-4efa-897a-fdf4b1cf1204 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:46.459927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25623e4a-398a-4efa-897a-fdf4b1cf1204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:46.468944 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25623e4a-398a-4efa-897a-fdf4b1cf1204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:46.470365 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25623e4a-398a-4efa-897a-fdf4b1cf1204 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:46.479631 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1438/2841] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:08:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:08:46.890029 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373f733-2d65-4e4f-9af4-862f94ac2842 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:46.891501 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-3373f733-2d65-4e4f-9af4-862f94ac2842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:46.907387 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373f733-2d65-4e4f-9af4-862f94ac2842 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:46.912346 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:46.916511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:46.916511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:46.916511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56 admin admin] PATH_INFO: `/v3/projects/9d07ad04f5a54f92a3b9ccc5fee960c6` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:46.916511 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d07ad04f5a54f92a3b9ccc5fee960c6)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:46.916950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:47.054564 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d07ad04f5a54f92a3b9ccc5fee960c6', 'request_id': 'req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:47.060612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d07ad04f5a54f92a3b9ccc5fee960c6', 'request_id': 'req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:47.066675 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d07ad04f5a54f92a3b9ccc5fee960c6', 'request_id': 'req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:47.133767 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56 admin admin] The token cache is being invalidate because project 9d07ad04f5a54f92a3b9ccc5fee960c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:08:47.134053 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:47.134668 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:47.136156 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1404/2842] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:08:46 2022] DELETE /identity/v3/projects/9d07ad04f5a54f92a3b9ccc5fee960c6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:47.140129 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25623e4a-398a-4efa-897a-fdf4b1cf1204 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:47.149670 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25623e4a-398a-4efa-897a-fdf4b1cf1204 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:47.168455 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:47.205681 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dc8ec-fb74-4e32-b821-1465a6459ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:47.206342 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dc8ec-fb74-4e32-b821-1465a6459ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:47.206622 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dc8ec-fb74-4e32-b821-1465a6459ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:47.206901 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dc8ec-fb74-4e32-b821-1465a6459ee0 admin admin] PATH_INFO: `/v3/users/c92b5a5d73594fd595411d45861eeceb` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:47.207999 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dc8ec-fb74-4e32-b821-1465a6459ee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c92b5a5d73594fd595411d45861eeceb)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:47.209950 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dc8ec-fb74-4e32-b821-1465a6459ee0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:47.381607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0a9dc8ec-fb74-4e32-b821-1465a6459ee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c92b5a5d73594fd595411d45861eeceb'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:47.382430 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0a9dc8ec-fb74-4e32-b821-1465a6459ee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c92b5a5d73594fd595411d45861eeceb'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:47.382879 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0a9dc8ec-fb74-4e32-b821-1465a6459ee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c92b5a5d73594fd595411d45861eeceb'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:47.453312 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0a9dc8ec-fb74-4e32-b821-1465a6459ee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c92b5a5d73594fd595411d45861eeceb'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:47.460753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0a9dc8ec-fb74-4e32-b821-1465a6459ee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c92b5a5d73594fd595411d45861eeceb'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:47.478644 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1439/2843] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:08:47 2022] DELETE /identity/v3/users/c92b5a5d73594fd595411d45861eeceb => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:47.902921 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:47.904482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:47.917932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8cde9-7fd4-4a2f-bcf9-5a56a90e5d56 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:47.975320 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb776f4-7ea2-4cf3-bccb-2cb682b59c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:47.975943 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb776f4-7ea2-4cf3-bccb-2cb682b59c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:47.976219 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb776f4-7ea2-4cf3-bccb-2cb682b59c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:47.976523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb776f4-7ea2-4cf3-bccb-2cb682b59c89 admin admin] PATH_INFO: `/v3/projects/519ce96682774cbf871033507b0d3fce` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:47.977621 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb776f4-7ea2-4cf3-bccb-2cb682b59c89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=519ce96682774cbf871033507b0d3fce)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:47.979507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb776f4-7ea2-4cf3-bccb-2cb682b59c89 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:48.084830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:48.100141 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2fb776f4-7ea2-4cf3-bccb-2cb682b59c89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '519ce96682774cbf871033507b0d3fce', 'request_id': 'req-2fb776f4-7ea2-4cf3-bccb-2cb682b59c89'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:48.106162 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2fb776f4-7ea2-4cf3-bccb-2cb682b59c89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '519ce96682774cbf871033507b0d3fce', 'request_id': 'req-2fb776f4-7ea2-4cf3-bccb-2cb682b59c89'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:48.111546 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2fb776f4-7ea2-4cf3-bccb-2cb682b59c89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '519ce96682774cbf871033507b0d3fce', 'request_id': 'req-2fb776f4-7ea2-4cf3-bccb-2cb682b59c89'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:48.179366 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2fb776f4-7ea2-4cf3-bccb-2cb682b59c89 admin admin] The token cache is being invalidate because project 519ce96682774cbf871033507b0d3fce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:08:48.179675 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2fb776f4-7ea2-4cf3-bccb-2cb682b59c89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:48.180311 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-2fb776f4-7ea2-4cf3-bccb-2cb682b59c89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:48.181807 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1405/2844] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:08:47 2022] DELETE /identity/v3/projects/519ce96682774cbf871033507b0d3fce => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:48.186785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59360506-9705-4791-a195-28fa9d39d058 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.187286 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59360506-9705-4791-a195-28fa9d39d058 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.187418 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59360506-9705-4791-a195-28fa9d39d058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.208446 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-59360506-9705-4791-a195-28fa9d39d058 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:48.209644 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-59360506-9705-4791-a195-28fa9d39d058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:48.266609 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1440/2845] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:48.326182 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a891e3-95b8-45d0-95e9-3af72e4cc2e1 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.326454 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a891e3-95b8-45d0-95e9-3af72e4cc2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.326738 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a891e3-95b8-45d0-95e9-3af72e4cc2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.347254 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-89a891e3-95b8-45d0-95e9-3af72e4cc2e1 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:48.348491 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-89a891e3-95b8-45d0-95e9-3af72e4cc2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:48.356294 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1406/2846] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:48.360465 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59360506-9705-4791-a195-28fa9d39d058 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:48.370905 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59360506-9705-4791-a195-28fa9d39d058 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:48.375430 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c513864-dcdc-4dd2-b3b8-3a0eb73bda36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:48.376022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c513864-dcdc-4dd2-b3b8-3a0eb73bda36 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.376300 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c513864-dcdc-4dd2-b3b8-3a0eb73bda36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.376857 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c513864-dcdc-4dd2-b3b8-3a0eb73bda36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.377618 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c513864-dcdc-4dd2-b3b8-3a0eb73bda36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:48.377925 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c513864-dcdc-4dd2-b3b8-3a0eb73bda36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:48.380111 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c513864-dcdc-4dd2-b3b8-3a0eb73bda36 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:48.391386 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1441/2847] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 17:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:48.395610 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a891e3-95b8-45d0-95e9-3af72e4cc2e1 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:48.404737 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a891e3-95b8-45d0-95e9-3af72e4cc2e1 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:48.413527 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ade36-e448-42db-a398-428bed576348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:48.414136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ade36-e448-42db-a398-428bed576348 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.414414 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ade36-e448-42db-a398-428bed576348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.414715 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ade36-e448-42db-a398-428bed576348 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.415347 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ade36-e448-42db-a398-428bed576348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:48.417298 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ade36-e448-42db-a398-428bed576348 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:48.495964 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1407/2848] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:08:48 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:48.500063 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c513864-dcdc-4dd2-b3b8-3a0eb73bda36 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:48.510967 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c513864-dcdc-4dd2-b3b8-3a0eb73bda36 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:48.535843 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:48.566932 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1f664-f0b7-4a31-b845-32740560c048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:48.567549 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1f664-f0b7-4a31-b845-32740560c048 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.567835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1f664-f0b7-4a31-b845-32740560c048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.568111 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1f664-f0b7-4a31-b845-32740560c048 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.568820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1f664-f0b7-4a31-b845-32740560c048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:48.570744 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1f664-f0b7-4a31-b845-32740560c048 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:48.665835 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1442/2849] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:08:48 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:48.669956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ade36-e448-42db-a398-428bed576348 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:48.681115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ade36-e448-42db-a398-428bed576348 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:48.685790 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016b582-25eb-410b-a0a3-81e7c02f4ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:48.686381 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016b582-25eb-410b-a0a3-81e7c02f4ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.686660 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016b582-25eb-410b-a0a3-81e7c02f4ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.686932 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016b582-25eb-410b-a0a3-81e7c02f4ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.687612 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016b582-25eb-410b-a0a3-81e7c02f4ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:48.689268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016b582-25eb-410b-a0a3-81e7c02f4ce9 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:48.699552 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1408/2850] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:48.703486 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1f664-f0b7-4a31-b845-32740560c048 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:48.713390 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1f664-f0b7-4a31-b845-32740560c048 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:48.717980 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e36ce-eba2-444c-a1ba-7132dd5b6459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:48.718571 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e36ce-eba2-444c-a1ba-7132dd5b6459 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.718852 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e36ce-eba2-444c-a1ba-7132dd5b6459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.719128 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e36ce-eba2-444c-a1ba-7132dd5b6459 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.719764 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e36ce-eba2-444c-a1ba-7132dd5b6459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:48.721413 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e36ce-eba2-444c-a1ba-7132dd5b6459 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:48.732159 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1443/2851] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:08:48.736161 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016b582-25eb-410b-a0a3-81e7c02f4ce9 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:48.746585 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016b582-25eb-410b-a0a3-81e7c02f4ce9 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:48.751072 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7bf69-84b6-4993-b41b-2d2b1b7ca743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:48.751961 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7bf69-84b6-4993-b41b-2d2b1b7ca743 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.752240 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7bf69-84b6-4993-b41b-2d2b1b7ca743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.752553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7bf69-84b6-4993-b41b-2d2b1b7ca743 admin admin] PATH_INFO: `/v3/projects/4186dd33aed743a9ad64d471c091ffc3/users/efdb50231fdb4e29becec37846f3524b/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.766553 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7bf69-84b6-4993-b41b-2d2b1b7ca743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4186dd33aed743a9ad64d471c091ffc3, user_id=efdb50231fdb4e29becec37846f3524b, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:48.769214 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7bf69-84b6-4993-b41b-2d2b1b7ca743 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:48.842737 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1409/2852] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:08:48 2022] PUT /identity/v3/projects/4186dd33aed743a9ad64d471c091ffc3/users/efdb50231fdb4e29becec37846f3524b/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:48.853918 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071af8f8-089a-4d7a-a09e-692919f19b18 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.854323 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071af8f8-089a-4d7a-a09e-692919f19b18 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.854675 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071af8f8-089a-4d7a-a09e-692919f19b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.883309 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-071af8f8-089a-4d7a-a09e-692919f19b18 None None] MFA Rules not processed for user `efdb50231fdb4e29becec37846f3524b`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:48.884595 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-071af8f8-089a-4d7a-a09e-692919f19b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:48.947110 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1444/2853] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:08:48.951818 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7bf69-84b6-4993-b41b-2d2b1b7ca743 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.012419 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7bf69-84b6-4993-b41b-2d2b1b7ca743 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.016914 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f55ff-aa85-471c-826b-83359b753c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.017507 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f55ff-aa85-471c-826b-83359b753c0c admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.017782 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f55ff-aa85-471c-826b-83359b753c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.018049 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f55ff-aa85-471c-826b-83359b753c0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.019172 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f55ff-aa85-471c-826b-83359b753c0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.021064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f55ff-aa85-471c-826b-83359b753c0c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.095128 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1410/2854] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:08:48 2022] POST /identity/v3/projects => generated 369 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:49.100093 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071af8f8-089a-4d7a-a09e-692919f19b18 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.110585 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071af8f8-089a-4d7a-a09e-692919f19b18 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.174974 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb65f7-0016-49fb-8897-5998d35b1d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.175581 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb65f7-0016-49fb-8897-5998d35b1d80 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.175861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb65f7-0016-49fb-8897-5998d35b1d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.176134 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb65f7-0016-49fb-8897-5998d35b1d80 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.176792 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb65f7-0016-49fb-8897-5998d35b1d80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.178711 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb65f7-0016-49fb-8897-5998d35b1d80 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.242790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1445/2855] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:08:49 2022] POST /identity/v3/users => generated 291 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:49.246863 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f55ff-aa85-471c-826b-83359b753c0c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.258492 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f55ff-aa85-471c-826b-83359b753c0c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.263232 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fefc97-032f-4d11-bb17-df6466e68bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.263771 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fefc97-032f-4d11-bb17-df6466e68bee admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.264051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fefc97-032f-4d11-bb17-df6466e68bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.264325 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fefc97-032f-4d11-bb17-df6466e68bee admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.264992 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fefc97-032f-4d11-bb17-df6466e68bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.266580 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fefc97-032f-4d11-bb17-df6466e68bee admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.277294 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1411/2856] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:49.281394 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb65f7-0016-49fb-8897-5998d35b1d80 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.291917 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb65f7-0016-49fb-8897-5998d35b1d80 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.296607 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4851d6-195b-4abd-bf00-d85043fb46ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.297237 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4851d6-195b-4abd-bf00-d85043fb46ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.297520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4851d6-195b-4abd-bf00-d85043fb46ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.297791 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4851d6-195b-4abd-bf00-d85043fb46ab admin admin] PATH_INFO: `/v3/projects/f3f71a5793484dbb9d2dc9e4213640ab/users/3e375b63834e45e2b8d93178f5e256ba/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.311995 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4851d6-195b-4abd-bf00-d85043fb46ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f71a5793484dbb9d2dc9e4213640ab, user_id=3e375b63834e45e2b8d93178f5e256ba, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.314657 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4851d6-195b-4abd-bf00-d85043fb46ab admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.373890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1446/2857] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:08:49 2022] PUT /identity/v3/projects/f3f71a5793484dbb9d2dc9e4213640ab/users/3e375b63834e45e2b8d93178f5e256ba/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:49.378638 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fefc97-032f-4d11-bb17-df6466e68bee admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.388554 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fefc97-032f-4d11-bb17-df6466e68bee admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.408210 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:49.443442 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b649e-5af1-45cd-9d30-d8ead980669f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.444079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b649e-5af1-45cd-9d30-d8ead980669f admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.444352 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b649e-5af1-45cd-9d30-d8ead980669f admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.444651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b649e-5af1-45cd-9d30-d8ead980669f admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.445284 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31b649e-5af1-45cd-9d30-d8ead980669f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.446878 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31b649e-5af1-45cd-9d30-d8ead980669f admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.456894 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1412/2858] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:49.461019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4851d6-195b-4abd-bf00-d85043fb46ab admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.466419 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:49.471880 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4851d6-195b-4abd-bf00-d85043fb46ab admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.494650 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b418245-985f-44fa-b7e4-44392dc42761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.495337 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b418245-985f-44fa-b7e4-44392dc42761 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.495623 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b418245-985f-44fa-b7e4-44392dc42761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.495916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b418245-985f-44fa-b7e4-44392dc42761 admin admin] PATH_INFO: `/v3/projects/f3f71a5793484dbb9d2dc9e4213640ab/users/3e375b63834e45e2b8d93178f5e256ba/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.497991 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b418245-985f-44fa-b7e4-44392dc42761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f71a5793484dbb9d2dc9e4213640ab, user_id=3e375b63834e45e2b8d93178f5e256ba, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.503820 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b418245-985f-44fa-b7e4-44392dc42761 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.516033 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1447/2859] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:08:49 2022] PUT /identity/v3/projects/f3f71a5793484dbb9d2dc9e4213640ab/users/3e375b63834e45e2b8d93178f5e256ba/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:49.527230 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b903b-bdaa-4c52-a4ed-e2a0047aa587 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.527515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b903b-bdaa-4c52-a4ed-e2a0047aa587 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.527787 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b903b-bdaa-4c52-a4ed-e2a0047aa587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.556079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-411b903b-bdaa-4c52-a4ed-e2a0047aa587 None None] MFA Rules not processed for user `3e375b63834e45e2b8d93178f5e256ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:49.557344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-411b903b-bdaa-4c52-a4ed-e2a0047aa587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:49.626212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1413/2860] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:08:49.638368 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b418245-985f-44fa-b7e4-44392dc42761 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.640056 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-1b418245-985f-44fa-b7e4-44392dc42761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:49.698660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b418245-985f-44fa-b7e4-44392dc42761 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.702421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9ecba-c778-444c-87af-51e207293398 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.703026 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9ecba-c778-444c-87af-51e207293398 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.703310 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9ecba-c778-444c-87af-51e207293398 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.703584 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9ecba-c778-444c-87af-51e207293398 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.712763 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9ecba-c778-444c-87af-51e207293398 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.715080 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9ecba-c778-444c-87af-51e207293398 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.723470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1448/2861] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:08:49.852173 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b903b-bdaa-4c52-a4ed-e2a0047aa587 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.853672 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-411b903b-bdaa-4c52-a4ed-e2a0047aa587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:49.866286 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b903b-bdaa-4c52-a4ed-e2a0047aa587 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.869885 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ca5fd-270b-49cc-9775-2c9d6437353f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.870481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ca5fd-270b-49cc-9775-2c9d6437353f service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.870765 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ca5fd-270b-49cc-9775-2c9d6437353f service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.871040 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ca5fd-270b-49cc-9775-2c9d6437353f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.926863 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ca5fd-270b-49cc-9775-2c9d6437353f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.928321 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ca5fd-270b-49cc-9775-2c9d6437353f service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.937254 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1414/2862] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:08:54.617120 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9ecba-c778-444c-87af-51e207293398 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:54.676991 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9ecba-c778-444c-87af-51e207293398 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:54.681451 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa900fca-6eaa-4795-a6bb-16464e4f24da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:54.682065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa900fca-6eaa-4795-a6bb-16464e4f24da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:54.682340 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa900fca-6eaa-4795-a6bb-16464e4f24da admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:54.682609 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa900fca-6eaa-4795-a6bb-16464e4f24da admin admin] PATH_INFO: `/v3/users/efdb50231fdb4e29becec37846f3524b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:54.683699 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa900fca-6eaa-4795-a6bb-16464e4f24da admin admin] RBAC: Authorizing `identity:delete_user(user_id=efdb50231fdb4e29becec37846f3524b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:54.685625 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa900fca-6eaa-4795-a6bb-16464e4f24da admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:54.856765 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fa900fca-6eaa-4795-a6bb-16464e4f24da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efdb50231fdb4e29becec37846f3524b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:54.857425 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fa900fca-6eaa-4795-a6bb-16464e4f24da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efdb50231fdb4e29becec37846f3524b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:54.858047 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fa900fca-6eaa-4795-a6bb-16464e4f24da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efdb50231fdb4e29becec37846f3524b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:54.927948 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fa900fca-6eaa-4795-a6bb-16464e4f24da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efdb50231fdb4e29becec37846f3524b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:54.934984 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fa900fca-6eaa-4795-a6bb-16464e4f24da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efdb50231fdb4e29becec37846f3524b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:54.952139 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1449/2863] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:08:54 2022] DELETE /identity/v3/users/efdb50231fdb4e29becec37846f3524b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:54.957256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6049531-cbec-4605-9738-d0bc4a215f6b None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:54.957741 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6049531-cbec-4605-9738-d0bc4a215f6b None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:54.957849 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6049531-cbec-4605-9738-d0bc4a215f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:54.977221 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-b6049531-cbec-4605-9738-d0bc4a215f6b None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:54.978460 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b6049531-cbec-4605-9738-d0bc4a215f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:55.017716 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:55.037245 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1415/2864] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:55.046981 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa900fca-6eaa-4795-a6bb-16464e4f24da admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:55.048388 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fa900fca-6eaa-4795-a6bb-16464e4f24da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:55.061731 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:55.105085 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa900fca-6eaa-4795-a6bb-16464e4f24da admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:55.108707 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb068839-723e-4d01-a61f-380535aef99b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:55.109246 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb068839-723e-4d01-a61f-380535aef99b service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:55.109520 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb068839-723e-4d01-a61f-380535aef99b service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:55.109822 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb068839-723e-4d01-a61f-380535aef99b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:55.119151 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb068839-723e-4d01-a61f-380535aef99b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:55.120587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb068839-723e-4d01-a61f-380535aef99b service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:55.129923 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1450/2865] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:08:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:08:55.528882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6049531-cbec-4605-9738-d0bc4a215f6b None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:55.528882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-b6049531-cbec-4605-9738-d0bc4a215f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:55.537613 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6049531-cbec-4605-9738-d0bc4a215f6b None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:55.549322 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cff1c-f78b-4b15-9645-7d1d74353d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:55.549956 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cff1c-f78b-4b15-9645-7d1d74353d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:55.550109 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cff1c-f78b-4b15-9645-7d1d74353d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:55.550392 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cff1c-f78b-4b15-9645-7d1d74353d55 admin admin] PATH_INFO: `/v3/projects/4186dd33aed743a9ad64d471c091ffc3` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:55.551504 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cff1c-f78b-4b15-9645-7d1d74353d55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4186dd33aed743a9ad64d471c091ffc3)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:55.553465 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cff1c-f78b-4b15-9645-7d1d74353d55 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:55.659158 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c48cff1c-f78b-4b15-9645-7d1d74353d55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4186dd33aed743a9ad64d471c091ffc3', 'request_id': 'req-c48cff1c-f78b-4b15-9645-7d1d74353d55'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:55.665148 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c48cff1c-f78b-4b15-9645-7d1d74353d55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4186dd33aed743a9ad64d471c091ffc3', 'request_id': 'req-c48cff1c-f78b-4b15-9645-7d1d74353d55'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:55.670836 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c48cff1c-f78b-4b15-9645-7d1d74353d55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4186dd33aed743a9ad64d471c091ffc3', 'request_id': 'req-c48cff1c-f78b-4b15-9645-7d1d74353d55'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:55.744845 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c48cff1c-f78b-4b15-9645-7d1d74353d55 admin admin] The token cache is being invalidate because project 4186dd33aed743a9ad64d471c091ffc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:08:55.745133 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c48cff1c-f78b-4b15-9645-7d1d74353d55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:55.745760 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-c48cff1c-f78b-4b15-9645-7d1d74353d55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:55.747234 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1416/2866] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:08:55 2022] DELETE /identity/v3/projects/4186dd33aed743a9ad64d471c091ffc3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:55.751171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb068839-723e-4d01-a61f-380535aef99b service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:55.761567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb068839-723e-4d01-a61f-380535aef99b service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:55.815771 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec6c67-00b8-457b-899a-97a26c4af402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:55.817039 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec6c67-00b8-457b-899a-97a26c4af402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:55.817314 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec6c67-00b8-457b-899a-97a26c4af402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:55.817587 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec6c67-00b8-457b-899a-97a26c4af402 admin admin] PATH_INFO: `/v3/users/3e375b63834e45e2b8d93178f5e256ba` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:55.818797 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec6c67-00b8-457b-899a-97a26c4af402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e375b63834e45e2b8d93178f5e256ba)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:55.820747 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec6c67-00b8-457b-899a-97a26c4af402 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:55.993311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9eec6c67-00b8-457b-899a-97a26c4af402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e375b63834e45e2b8d93178f5e256ba'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:55.993902 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9eec6c67-00b8-457b-899a-97a26c4af402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e375b63834e45e2b8d93178f5e256ba'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:55.994556 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9eec6c67-00b8-457b-899a-97a26c4af402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e375b63834e45e2b8d93178f5e256ba'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:56.065091 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9eec6c67-00b8-457b-899a-97a26c4af402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e375b63834e45e2b8d93178f5e256ba'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:56.072211 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9eec6c67-00b8-457b-899a-97a26c4af402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e375b63834e45e2b8d93178f5e256ba'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:56.089349 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1451/2867] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:08:55 2022] DELETE /identity/v3/users/3e375b63834e45e2b8d93178f5e256ba => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:56.483747 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cff1c-f78b-4b15-9645-7d1d74353d55 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:56.485223 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-c48cff1c-f78b-4b15-9645-7d1d74353d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:56.498896 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cff1c-f78b-4b15-9645-7d1d74353d55 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:56.557177 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00ee8d-bb42-4375-8ed0-7d28121c946c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:56.557813 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00ee8d-bb42-4375-8ed0-7d28121c946c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:56.558097 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00ee8d-bb42-4375-8ed0-7d28121c946c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:56.558371 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00ee8d-bb42-4375-8ed0-7d28121c946c admin admin] PATH_INFO: `/v3/projects/f3f71a5793484dbb9d2dc9e4213640ab` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:56.559608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00ee8d-bb42-4375-8ed0-7d28121c946c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3f71a5793484dbb9d2dc9e4213640ab)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:56.561594 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00ee8d-bb42-4375-8ed0-7d28121c946c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:56.659045 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fb00ee8d-bb42-4375-8ed0-7d28121c946c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3f71a5793484dbb9d2dc9e4213640ab', 'request_id': 'req-fb00ee8d-bb42-4375-8ed0-7d28121c946c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:56.665296 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fb00ee8d-bb42-4375-8ed0-7d28121c946c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3f71a5793484dbb9d2dc9e4213640ab', 'request_id': 'req-fb00ee8d-bb42-4375-8ed0-7d28121c946c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:56.670833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fb00ee8d-bb42-4375-8ed0-7d28121c946c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3f71a5793484dbb9d2dc9e4213640ab', 'request_id': 'req-fb00ee8d-bb42-4375-8ed0-7d28121c946c'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:56.723817 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:56.728417 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fb00ee8d-bb42-4375-8ed0-7d28121c946c admin admin] The token cache is being invalidate because project f3f71a5793484dbb9d2dc9e4213640ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:08:56.728722 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fb00ee8d-bb42-4375-8ed0-7d28121c946c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:56.729344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-fb00ee8d-bb42-4375-8ed0-7d28121c946c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:56.730843 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1417/2868] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:08:56 2022] DELETE /identity/v3/projects/f3f71a5793484dbb9d2dc9e4213640ab => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:56.735806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc56c56-86b6-45ad-a68a-fcd1b6f58fb5 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:56.736119 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc56c56-86b6-45ad-a68a-fcd1b6f58fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:56.736417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc56c56-86b6-45ad-a68a-fcd1b6f58fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:56.756430 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-1fc56c56-86b6-45ad-a68a-fcd1b6f58fb5 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:56.757708 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-1fc56c56-86b6-45ad-a68a-fcd1b6f58fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:56.786453 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:56.815803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1452/2869] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:56.824037 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e468a-0a23-436f-b0b6-b6c1cf02e88d None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:56.824319 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e468a-0a23-436f-b0b6-b6c1cf02e88d None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:56.824734 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e468a-0a23-436f-b0b6-b6c1cf02e88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:56.844844 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-f54e468a-0a23-436f-b0b6-b6c1cf02e88d None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:56.846035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-f54e468a-0a23-436f-b0b6-b6c1cf02e88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:56.854007 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1418/2870] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:56.858232 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc56c56-86b6-45ad-a68a-fcd1b6f58fb5 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:56.867813 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc56c56-86b6-45ad-a68a-fcd1b6f58fb5 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:56.872350 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3766e03-6cd9-4ddc-92ca-0f8e4851736c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:56.872970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3766e03-6cd9-4ddc-92ca-0f8e4851736c admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:56.873254 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3766e03-6cd9-4ddc-92ca-0f8e4851736c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:56.873530 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3766e03-6cd9-4ddc-92ca-0f8e4851736c admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:56.874178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3766e03-6cd9-4ddc-92ca-0f8e4851736c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:56.874483 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3766e03-6cd9-4ddc-92ca-0f8e4851736c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:56.876144 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3766e03-6cd9-4ddc-92ca-0f8e4851736c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:56.888838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1453/2871] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 17:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:56.893122 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e468a-0a23-436f-b0b6-b6c1cf02e88d None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:56.902749 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e468a-0a23-436f-b0b6-b6c1cf02e88d None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:56.908136 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f0718-4caa-4558-9b86-8daa2f19f338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:56.909799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f0718-4caa-4558-9b86-8daa2f19f338 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:56.909799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f0718-4caa-4558-9b86-8daa2f19f338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:56.909799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f0718-4caa-4558-9b86-8daa2f19f338 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:56.910131 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f0718-4caa-4558-9b86-8daa2f19f338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:56.912482 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f0718-4caa-4558-9b86-8daa2f19f338 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:56.986881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1419/2872] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:08:56 2022] POST /identity/v3/projects => generated 381 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:08:56.991228 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3766e03-6cd9-4ddc-92ca-0f8e4851736c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:57.001294 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3766e03-6cd9-4ddc-92ca-0f8e4851736c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:57.057545 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232db1a-cd31-4aad-b4be-9b4d645ada39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:57.058071 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232db1a-cd31-4aad-b4be-9b4d645ada39 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.058380 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232db1a-cd31-4aad-b4be-9b4d645ada39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.058653 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232db1a-cd31-4aad-b4be-9b4d645ada39 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.059276 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232db1a-cd31-4aad-b4be-9b4d645ada39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:57.061208 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232db1a-cd31-4aad-b4be-9b4d645ada39 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:57.162458 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1454/2873] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:08:56 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:57.171650 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f0718-4caa-4558-9b86-8daa2f19f338 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:57.182712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f0718-4caa-4558-9b86-8daa2f19f338 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:57.189772 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c6a60-da33-4264-bb12-930a0d0274b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:57.190378 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c6a60-da33-4264-bb12-930a0d0274b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.190663 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c6a60-da33-4264-bb12-930a0d0274b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.190939 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c6a60-da33-4264-bb12-930a0d0274b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.191586 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c6a60-da33-4264-bb12-930a0d0274b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:57.193808 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c6a60-da33-4264-bb12-930a0d0274b4 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:57.203826 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1420/2874] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:57.207970 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232db1a-cd31-4aad-b4be-9b4d645ada39 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:57.217880 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232db1a-cd31-4aad-b4be-9b4d645ada39 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:57.222383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53629a7-db52-4e3a-b752-f2bee298b9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:57.222993 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53629a7-db52-4e3a-b752-f2bee298b9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.223272 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53629a7-db52-4e3a-b752-f2bee298b9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.223542 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53629a7-db52-4e3a-b752-f2bee298b9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.224171 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53629a7-db52-4e3a-b752-f2bee298b9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:57.225776 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53629a7-db52-4e3a-b752-f2bee298b9e0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:57.235938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1455/2875] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:57.240300 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c6a60-da33-4264-bb12-930a0d0274b4 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:57.250115 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c6a60-da33-4264-bb12-930a0d0274b4 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:57.255613 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad9161-0d6d-4cb8-b105-a52b02977368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:57.256248 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad9161-0d6d-4cb8-b105-a52b02977368 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.256571 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad9161-0d6d-4cb8-b105-a52b02977368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.256849 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad9161-0d6d-4cb8-b105-a52b02977368 admin admin] PATH_INFO: `/v3/projects/c397d0acbbb2425d94125ae0e49cd91b/users/6cf4a39b606745169e247571f59f421a/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.270909 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad9161-0d6d-4cb8-b105-a52b02977368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c397d0acbbb2425d94125ae0e49cd91b, user_id=6cf4a39b606745169e247571f59f421a, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:57.273608 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad9161-0d6d-4cb8-b105-a52b02977368 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:57.339243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1421/2876] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:08:57 2022] PUT /identity/v3/projects/c397d0acbbb2425d94125ae0e49cd91b/users/6cf4a39b606745169e247571f59f421a/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:57.349260 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa183f8a-26f9-4612-a07f-4fb11c745309 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.349543 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa183f8a-26f9-4612-a07f-4fb11c745309 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.349855 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa183f8a-26f9-4612-a07f-4fb11c745309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.377170 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fa183f8a-26f9-4612-a07f-4fb11c745309 None None] MFA Rules not processed for user `6cf4a39b606745169e247571f59f421a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:57.378349 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fa183f8a-26f9-4612-a07f-4fb11c745309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:57.439070 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1456/2877] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:57.444146 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad9161-0d6d-4cb8-b105-a52b02977368 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:57.505182 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad9161-0d6d-4cb8-b105-a52b02977368 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:57.509912 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c032b7-e8f4-457c-919e-f2596d8bfcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:57.510508 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c032b7-e8f4-457c-919e-f2596d8bfcc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.510788 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c032b7-e8f4-457c-919e-f2596d8bfcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.511064 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c032b7-e8f4-457c-919e-f2596d8bfcc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.511680 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c032b7-e8f4-457c-919e-f2596d8bfcc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:57.513613 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c032b7-e8f4-457c-919e-f2596d8bfcc1 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:57.592762 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1422/2878] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:08:57 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:57.596853 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa183f8a-26f9-4612-a07f-4fb11c745309 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:57.608004 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa183f8a-26f9-4612-a07f-4fb11c745309 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:57.633845 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:57.663188 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fca85e-68eb-4481-8470-623d695970de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:57.663786 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fca85e-68eb-4481-8470-623d695970de admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.664070 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fca85e-68eb-4481-8470-623d695970de admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.664342 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fca85e-68eb-4481-8470-623d695970de admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.664997 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fca85e-68eb-4481-8470-623d695970de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:57.666897 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fca85e-68eb-4481-8470-623d695970de admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:57.745693 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1457/2879] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:08:57 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:57.749694 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c032b7-e8f4-457c-919e-f2596d8bfcc1 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:57.760252 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c032b7-e8f4-457c-919e-f2596d8bfcc1 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:57.765100 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b9ba7-3749-4629-a546-5e8b9dae87cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:57.765696 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b9ba7-3749-4629-a546-5e8b9dae87cf admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.765974 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b9ba7-3749-4629-a546-5e8b9dae87cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.766245 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b9ba7-3749-4629-a546-5e8b9dae87cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.766911 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b9ba7-3749-4629-a546-5e8b9dae87cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:57.768529 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b9ba7-3749-4629-a546-5e8b9dae87cf admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:57.779016 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1423/2880] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:57.782985 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fca85e-68eb-4481-8470-623d695970de admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:57.793452 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fca85e-68eb-4481-8470-623d695970de admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:57.798016 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebc915-4079-47c3-ac50-a415484777c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:57.798660 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebc915-4079-47c3-ac50-a415484777c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.798943 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebc915-4079-47c3-ac50-a415484777c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.799219 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebc915-4079-47c3-ac50-a415484777c6 admin admin] PATH_INFO: `/v3/projects/cb6bf6fb9f9f40729024780fec134e36/users/42c515ccedd4452aa3388addfc45e0dc/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.814207 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ebc915-4079-47c3-ac50-a415484777c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6bf6fb9f9f40729024780fec134e36, user_id=42c515ccedd4452aa3388addfc45e0dc, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:57.816939 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ebc915-4079-47c3-ac50-a415484777c6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:57.865815 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1458/2881] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:08:57 2022] PUT /identity/v3/projects/cb6bf6fb9f9f40729024780fec134e36/users/42c515ccedd4452aa3388addfc45e0dc/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:08:57.869712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b9ba7-3749-4629-a546-5e8b9dae87cf admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:57.879830 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b9ba7-3749-4629-a546-5e8b9dae87cf admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:57.935515 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90b950-74d6-41d8-b5bf-f91e964c540e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:57.936116 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90b950-74d6-41d8-b5bf-f91e964c540e admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.936421 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90b950-74d6-41d8-b5bf-f91e964c540e admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.936699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90b950-74d6-41d8-b5bf-f91e964c540e admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.937331 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90b950-74d6-41d8-b5bf-f91e964c540e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:57.938971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90b950-74d6-41d8-b5bf-f91e964c540e admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:57.948957 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1424/2882] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:57.953152 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebc915-4079-47c3-ac50-a415484777c6 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:57.964152 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebc915-4079-47c3-ac50-a415484777c6 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:57.968861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ed3e7-4945-4874-bf2f-fd910efb4b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:57.969509 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ed3e7-4945-4874-bf2f-fd910efb4b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.969794 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ed3e7-4945-4874-bf2f-fd910efb4b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.970072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ed3e7-4945-4874-bf2f-fd910efb4b92 admin admin] PATH_INFO: `/v3/projects/cb6bf6fb9f9f40729024780fec134e36/users/42c515ccedd4452aa3388addfc45e0dc/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.971996 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ed3e7-4945-4874-bf2f-fd910efb4b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6bf6fb9f9f40729024780fec134e36, user_id=42c515ccedd4452aa3388addfc45e0dc, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:57.975321 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ed3e7-4945-4874-bf2f-fd910efb4b92 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:57.985422 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1459/2883] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:08:57 2022] PUT /identity/v3/projects/cb6bf6fb9f9f40729024780fec134e36/users/42c515ccedd4452aa3388addfc45e0dc/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:57.996825 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61521537-57c5-4dff-a0f7-a1d9d0b949a1 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:58.002549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61521537-57c5-4dff-a0f7-a1d9d0b949a1 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:58.002549 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61521537-57c5-4dff-a0f7-a1d9d0b949a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:58.005697 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:58.038596 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-61521537-57c5-4dff-a0f7-a1d9d0b949a1 None None] MFA Rules not processed for user `42c515ccedd4452aa3388addfc45e0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:58.039799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-61521537-57c5-4dff-a0f7-a1d9d0b949a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:58.107799 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1425/2884] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:08:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:08:58.120473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ed3e7-4945-4874-bf2f-fd910efb4b92 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:58.122249 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-525ed3e7-4945-4874-bf2f-fd910efb4b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:58.180790 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ed3e7-4945-4874-bf2f-fd910efb4b92 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:58.184429 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b3abc-8be1-420e-a35f-bade303ffe48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:58.185029 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b3abc-8be1-420e-a35f-bade303ffe48 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:58.185316 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b3abc-8be1-420e-a35f-bade303ffe48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:58.185591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b3abc-8be1-420e-a35f-bade303ffe48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:58.194978 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b3abc-8be1-420e-a35f-bade303ffe48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:58.196420 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b3abc-8be1-420e-a35f-bade303ffe48 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:58.205756 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1460/2885] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:08:58 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:08:59.067305 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61521537-57c5-4dff-a0f7-a1d9d0b949a1 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:59.076485 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61521537-57c5-4dff-a0f7-a1d9d0b949a1 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:59.081099 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c31d4-3775-4597-b812-db5f7b766df0 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c515ccedd4452aa3388addfc45e0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cb6bf6fb9f9f40729024780fec134e36', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:59.081699 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c31d4-3775-4597-b812-db5f7b766df0 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:59.081980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c31d4-3775-4597-b812-db5f7b766df0 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:59.082256 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c31d4-3775-4597-b812-db5f7b766df0 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:59.083532 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c31d4-3775-4597-b812-db5f7b766df0 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:59.083835 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c31d4-3775-4597-b812-db5f7b766df0 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:59.085456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c31d4-3775-4597-b812-db5f7b766df0 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:59.096763 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1426/2886] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 17:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:59.100906 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b3abc-8be1-420e-a35f-bade303ffe48 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:59.109753 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b3abc-8be1-420e-a35f-bade303ffe48 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:59.114283 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403192c4-559d-46aa-a506-9f500ddfb5f6 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c515ccedd4452aa3388addfc45e0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cb6bf6fb9f9f40729024780fec134e36', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:59.114868 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403192c4-559d-46aa-a506-9f500ddfb5f6 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:59.115147 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403192c4-559d-46aa-a506-9f500ddfb5f6 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:59.115419 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403192c4-559d-46aa-a506-9f500ddfb5f6 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:59.116038 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403192c4-559d-46aa-a506-9f500ddfb5f6 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:59.117930 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403192c4-559d-46aa-a506-9f500ddfb5f6 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:59.191142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1461/2887] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:08:59 2022] POST /identity/v3/projects => generated 355 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:59.195712 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c31d4-3775-4597-b812-db5f7b766df0 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:59.205979 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c31d4-3775-4597-b812-db5f7b766df0 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:59.262288 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0cb15-29cc-4975-93eb-07c6ae5e0857 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c515ccedd4452aa3388addfc45e0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cb6bf6fb9f9f40729024780fec134e36', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:59.262892 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0cb15-29cc-4975-93eb-07c6ae5e0857 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:59.263179 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0cb15-29cc-4975-93eb-07c6ae5e0857 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:59.263456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0cb15-29cc-4975-93eb-07c6ae5e0857 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:59.264087 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0cb15-29cc-4975-93eb-07c6ae5e0857 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:59.264523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0cb15-29cc-4975-93eb-07c6ae5e0857 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:59.266114 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0cb15-29cc-4975-93eb-07c6ae5e0857 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:59.278212 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1427/2888] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 17:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:59.314890 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403192c4-559d-46aa-a506-9f500ddfb5f6 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:59.325292 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403192c4-559d-46aa-a506-9f500ddfb5f6 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:59.329927 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51795f3-410d-4821-a878-42e8bd578bba tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c515ccedd4452aa3388addfc45e0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cb6bf6fb9f9f40729024780fec134e36', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:59.330552 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51795f3-410d-4821-a878-42e8bd578bba tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:59.330838 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51795f3-410d-4821-a878-42e8bd578bba tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:59.331115 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51795f3-410d-4821-a878-42e8bd578bba tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] PATH_INFO: `/v3/projects/6ce2cb989a134dccae3bfc15398580d8` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:59.332247 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51795f3-410d-4821-a878-42e8bd578bba tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6ce2cb989a134dccae3bfc15398580d8)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:59.335261 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51795f3-410d-4821-a878-42e8bd578bba tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:59.430333 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d51795f3-410d-4821-a878-42e8bd578bba tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce2cb989a134dccae3bfc15398580d8', 'request_id': 'req-d51795f3-410d-4821-a878-42e8bd578bba'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:59.436895 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d51795f3-410d-4821-a878-42e8bd578bba tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce2cb989a134dccae3bfc15398580d8', 'request_id': 'req-d51795f3-410d-4821-a878-42e8bd578bba'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:59.438909 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:59.443395 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d51795f3-410d-4821-a878-42e8bd578bba tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce2cb989a134dccae3bfc15398580d8', 'request_id': 'req-d51795f3-410d-4821-a878-42e8bd578bba'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:59.526916 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d51795f3-410d-4821-a878-42e8bd578bba tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] The token cache is being invalidate because project 6ce2cb989a134dccae3bfc15398580d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:08:59.527229 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d51795f3-410d-4821-a878-42e8bd578bba tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:59.527862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d51795f3-410d-4821-a878-42e8bd578bba tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:08:59.529377 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1462/2889] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:08:59 2022] DELETE /identity/v3/projects/6ce2cb989a134dccae3bfc15398580d8 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:09:00.079859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0cb15-29cc-4975-93eb-07c6ae5e0857 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:00.081631 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-4be0cb15-29cc-4975-93eb-07c6ae5e0857 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:00.147201 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0cb15-29cc-4975-93eb-07c6ae5e0857 tempest-VolumeQuotaClassesTest-634531854 tempest-VolumeQuotaClassesTest-634531854-project-admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:00.158992 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=100 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:00.202107 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e270a9d-a2c3-4542-8379-0e38a06eca31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:00.202730 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e270a9d-a2c3-4542-8379-0e38a06eca31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:00.203010 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e270a9d-a2c3-4542-8379-0e38a06eca31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:00.203285 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e270a9d-a2c3-4542-8379-0e38a06eca31 admin admin] PATH_INFO: `/v3/users/6cf4a39b606745169e247571f59f421a` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:00.204449 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e270a9d-a2c3-4542-8379-0e38a06eca31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cf4a39b606745169e247571f59f421a)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:00.206393 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e270a9d-a2c3-4542-8379-0e38a06eca31 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:00.387080 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-0e270a9d-a2c3-4542-8379-0e38a06eca31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cf4a39b606745169e247571f59f421a'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:00.387763 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-0e270a9d-a2c3-4542-8379-0e38a06eca31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cf4a39b606745169e247571f59f421a'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:00.388375 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-0e270a9d-a2c3-4542-8379-0e38a06eca31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cf4a39b606745169e247571f59f421a'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:00.395376 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-0e270a9d-a2c3-4542-8379-0e38a06eca31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cf4a39b606745169e247571f59f421a'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:00.477066 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-0e270a9d-a2c3-4542-8379-0e38a06eca31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cf4a39b606745169e247571f59f421a'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:00.495430 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1428/2890] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:09:00 2022] DELETE /identity/v3/users/6cf4a39b606745169e247571f59f421a => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:09:00.499931 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906507c-d065-4d5a-b784-09049903b5a4 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:00.500470 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906507c-d065-4d5a-b784-09049903b5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:00.500582 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906507c-d065-4d5a-b784-09049903b5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:00.519068 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c906507c-d065-4d5a-b784-09049903b5a4 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:00.520258 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c906507c-d065-4d5a-b784-09049903b5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:00.580019 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1463/2891] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:09:00.589644 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e270a9d-a2c3-4542-8379-0e38a06eca31 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:00.591028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-0e270a9d-a2c3-4542-8379-0e38a06eca31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:00.649438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e270a9d-a2c3-4542-8379-0e38a06eca31 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:00.652941 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32037fef-d2e6-4dc0-af13-1069ab25d0fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:00.653543 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32037fef-d2e6-4dc0-af13-1069ab25d0fc service neutron] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:00.653820 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32037fef-d2e6-4dc0-af13-1069ab25d0fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:00.654089 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32037fef-d2e6-4dc0-af13-1069ab25d0fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:00.663344 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32037fef-d2e6-4dc0-af13-1069ab25d0fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:00.664802 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32037fef-d2e6-4dc0-af13-1069ab25d0fc service neutron] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:00.674147 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1429/2892] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:09:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:09:01.061238 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906507c-d065-4d5a-b784-09049903b5a4 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:01.062702 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c906507c-d065-4d5a-b784-09049903b5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:01.074821 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906507c-d065-4d5a-b784-09049903b5a4 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:01.079809 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47809d67-e461-465a-a0b0-78aeb7c8894a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:01.080477 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47809d67-e461-465a-a0b0-78aeb7c8894a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:01.080761 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47809d67-e461-465a-a0b0-78aeb7c8894a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:01.081064 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47809d67-e461-465a-a0b0-78aeb7c8894a admin admin] PATH_INFO: `/v3/projects/c397d0acbbb2425d94125ae0e49cd91b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:01.082158 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47809d67-e461-465a-a0b0-78aeb7c8894a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c397d0acbbb2425d94125ae0e49cd91b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:01.084220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47809d67-e461-465a-a0b0-78aeb7c8894a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:01.202785 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-47809d67-e461-465a-a0b0-78aeb7c8894a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c397d0acbbb2425d94125ae0e49cd91b', 'request_id': 'req-47809d67-e461-465a-a0b0-78aeb7c8894a'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:01.209044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-47809d67-e461-465a-a0b0-78aeb7c8894a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c397d0acbbb2425d94125ae0e49cd91b', 'request_id': 'req-47809d67-e461-465a-a0b0-78aeb7c8894a'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:01.215079 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-47809d67-e461-465a-a0b0-78aeb7c8894a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c397d0acbbb2425d94125ae0e49cd91b', 'request_id': 'req-47809d67-e461-465a-a0b0-78aeb7c8894a'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:01.304518 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-47809d67-e461-465a-a0b0-78aeb7c8894a admin admin] The token cache is being invalidate because project c397d0acbbb2425d94125ae0e49cd91b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:09:01.304806 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-47809d67-e461-465a-a0b0-78aeb7c8894a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:01.305421 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-47809d67-e461-465a-a0b0-78aeb7c8894a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:01.306907 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1464/2893] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:09:01 2022] DELETE /identity/v3/projects/c397d0acbbb2425d94125ae0e49cd91b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:09:01.310951 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32037fef-d2e6-4dc0-af13-1069ab25d0fc service neutron] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:01.321289 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32037fef-d2e6-4dc0-af13-1069ab25d0fc service neutron] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:01.359634 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:01.377754 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e9003-29f4-4e55-a6a2-d7f72d4589a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:01.378397 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e9003-29f4-4e55-a6a2-d7f72d4589a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:01.378654 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e9003-29f4-4e55-a6a2-d7f72d4589a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:01.378925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e9003-29f4-4e55-a6a2-d7f72d4589a0 admin admin] PATH_INFO: `/v3/users/42c515ccedd4452aa3388addfc45e0dc` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:01.380026 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e9003-29f4-4e55-a6a2-d7f72d4589a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c515ccedd4452aa3388addfc45e0dc)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:01.382541 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e9003-29f4-4e55-a6a2-d7f72d4589a0 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:01.576035 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-bc1e9003-29f4-4e55-a6a2-d7f72d4589a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c515ccedd4452aa3388addfc45e0dc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:01.576737 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-bc1e9003-29f4-4e55-a6a2-d7f72d4589a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c515ccedd4452aa3388addfc45e0dc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:01.577365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-bc1e9003-29f4-4e55-a6a2-d7f72d4589a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c515ccedd4452aa3388addfc45e0dc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:01.584441 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-bc1e9003-29f4-4e55-a6a2-d7f72d4589a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c515ccedd4452aa3388addfc45e0dc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:01.647741 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-bc1e9003-29f4-4e55-a6a2-d7f72d4589a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c515ccedd4452aa3388addfc45e0dc'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:01.667231 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1430/2894] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:09:01 2022] DELETE /identity/v3/users/42c515ccedd4452aa3388addfc45e0dc => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:09:02.095489 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47809d67-e461-465a-a0b0-78aeb7c8894a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:02.096989 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-47809d67-e461-465a-a0b0-78aeb7c8894a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:02.110801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47809d67-e461-465a-a0b0-78aeb7c8894a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:02.167892 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae383c3-93a0-41c3-9537-4703bcb2410c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:02.168557 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae383c3-93a0-41c3-9537-4703bcb2410c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:02.168841 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae383c3-93a0-41c3-9537-4703bcb2410c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:02.169190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae383c3-93a0-41c3-9537-4703bcb2410c admin admin] PATH_INFO: `/v3/projects/cb6bf6fb9f9f40729024780fec134e36` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:02.170322 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae383c3-93a0-41c3-9537-4703bcb2410c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb6bf6fb9f9f40729024780fec134e36)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:02.172264 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae383c3-93a0-41c3-9537-4703bcb2410c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:02.183183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:02.277906 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-aae383c3-93a0-41c3-9537-4703bcb2410c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb6bf6fb9f9f40729024780fec134e36', 'request_id': 'req-aae383c3-93a0-41c3-9537-4703bcb2410c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:02.284187 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-aae383c3-93a0-41c3-9537-4703bcb2410c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb6bf6fb9f9f40729024780fec134e36', 'request_id': 'req-aae383c3-93a0-41c3-9537-4703bcb2410c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:02.290547 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-aae383c3-93a0-41c3-9537-4703bcb2410c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb6bf6fb9f9f40729024780fec134e36', 'request_id': 'req-aae383c3-93a0-41c3-9537-4703bcb2410c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:02.383667 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-aae383c3-93a0-41c3-9537-4703bcb2410c admin admin] The token cache is being invalidate because project cb6bf6fb9f9f40729024780fec134e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:09:02.383953 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-aae383c3-93a0-41c3-9537-4703bcb2410c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:02.384591 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-aae383c3-93a0-41c3-9537-4703bcb2410c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:02.386076 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1465/2895] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:09:02 2022] DELETE /identity/v3/projects/cb6bf6fb9f9f40729024780fec134e36 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:09:02.391013 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe86c64-191e-403d-a721-ed99d645b447 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:02.391523 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe86c64-191e-403d-a721-ed99d645b447 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:02.391637 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe86c64-191e-403d-a721-ed99d645b447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:02.411059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-0fe86c64-191e-403d-a721-ed99d645b447 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:02.412271 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-0fe86c64-191e-403d-a721-ed99d645b447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:02.469827 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1431/2896] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:09:02.478152 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780dc1d-dd37-4b39-9594-77dbedcb32a0 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:02.478445 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780dc1d-dd37-4b39-9594-77dbedcb32a0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:02.478728 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780dc1d-dd37-4b39-9594-77dbedcb32a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:02.497938 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9780dc1d-dd37-4b39-9594-77dbedcb32a0 None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:02.499142 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9780dc1d-dd37-4b39-9594-77dbedcb32a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:02.507241 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1466/2897] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:09:02.511268 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe86c64-191e-403d-a721-ed99d645b447 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:02.520899 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe86c64-191e-403d-a721-ed99d645b447 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:02.525859 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e04465-e263-48fa-a9e0-7ab18d89178c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:02.526446 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e04465-e263-48fa-a9e0-7ab18d89178c admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:02.526722 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e04465-e263-48fa-a9e0-7ab18d89178c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:02.526991 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e04465-e263-48fa-a9e0-7ab18d89178c admin admin] PATH_INFO: `/v3/domains` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:02.527628 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e04465-e263-48fa-a9e0-7ab18d89178c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:02.527925 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e04465-e263-48fa-a9e0-7ab18d89178c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:02.529522 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e04465-e263-48fa-a9e0-7ab18d89178c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:02.541438 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1432/2898] 10.222.0.70 () {58 vars in 1153 bytes} [Wed Aug 24 17:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:02.545774 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780dc1d-dd37-4b39-9594-77dbedcb32a0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:02.555125 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780dc1d-dd37-4b39-9594-77dbedcb32a0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:02.559861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc601a7-3fdf-4426-95d5-9bc86f772d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:02.560494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc601a7-3fdf-4426-95d5-9bc86f772d21 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:02.560910 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc601a7-3fdf-4426-95d5-9bc86f772d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:02.561190 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc601a7-3fdf-4426-95d5-9bc86f772d21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:02.561822 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc601a7-3fdf-4426-95d5-9bc86f772d21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:02.563839 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc601a7-3fdf-4426-95d5-9bc86f772d21 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:02.660044 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1467/2899] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:09:02 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:09:02.664424 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e04465-e263-48fa-a9e0-7ab18d89178c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:02.674620 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e04465-e263-48fa-a9e0-7ab18d89178c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:02.717803 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:02.730181 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d0781-3ddd-4738-9c60-f8dece663da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:02.730775 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d0781-3ddd-4738-9c60-f8dece663da6 admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:02.731055 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d0781-3ddd-4738-9c60-f8dece663da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:02.731329 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d0781-3ddd-4738-9c60-f8dece663da6 admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:02.731998 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d0781-3ddd-4738-9c60-f8dece663da6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:02.733934 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d0781-3ddd-4738-9c60-f8dece663da6 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:02.807881 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1433/2900] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:09:02 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:09:02.811954 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc601a7-3fdf-4426-95d5-9bc86f772d21 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:02.822290 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc601a7-3fdf-4426-95d5-9bc86f772d21 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:02.826968 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2e0af-20ab-46a5-b839-7b3787a340eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:02.827567 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2e0af-20ab-46a5-b839-7b3787a340eb admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:02.827849 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2e0af-20ab-46a5-b839-7b3787a340eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:02.828121 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2e0af-20ab-46a5-b839-7b3787a340eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:02.834200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2e0af-20ab-46a5-b839-7b3787a340eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:02.834200 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2e0af-20ab-46a5-b839-7b3787a340eb admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:02.840103 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1468/2901] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:09:02.843922 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d0781-3ddd-4738-9c60-f8dece663da6 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:02.854534 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d0781-3ddd-4738-9c60-f8dece663da6 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:02.859196 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f22063-1fba-4ef0-b376-1336cb23dc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:02.862950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f22063-1fba-4ef0-b376-1336cb23dc49 admin admin] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:02.862950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f22063-1fba-4ef0-b376-1336cb23dc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:02.862950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f22063-1fba-4ef0-b376-1336cb23dc49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:02.862950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f22063-1fba-4ef0-b376-1336cb23dc49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:02.862950 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f22063-1fba-4ef0-b376-1336cb23dc49 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:02.873569 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1434/2902] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:09:02.877913 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2e0af-20ab-46a5-b839-7b3787a340eb admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:02.887751 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2e0af-20ab-46a5-b839-7b3787a340eb admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:02.892355 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf9709-e22f-4af5-bde4-e4431ab7aee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:02.893046 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf9709-e22f-4af5-bde4-e4431ab7aee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:02.893331 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf9709-e22f-4af5-bde4-e4431ab7aee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:02.893611 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf9709-e22f-4af5-bde4-e4431ab7aee9 admin admin] PATH_INFO: `/v3/projects/361a04154f264f6195120fb1ff42ba5e/users/f8b44cf61337411cb343fd4423318006/roles/c7b826398b4e466491d4723321aaa5f1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:02.908235 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf9709-e22f-4af5-bde4-e4431ab7aee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=361a04154f264f6195120fb1ff42ba5e, user_id=f8b44cf61337411cb343fd4423318006, role_id=c7b826398b4e466491d4723321aaa5f1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:02.911053 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf9709-e22f-4af5-bde4-e4431ab7aee9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:02.997636 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1469/2903] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:09:02 2022] PUT /identity/v3/projects/361a04154f264f6195120fb1ff42ba5e/users/f8b44cf61337411cb343fd4423318006/roles/c7b826398b4e466491d4723321aaa5f1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:09:03.003307 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee52fb-e5d9-4bd7-9e3e-df56563b250e None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:03.003595 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee52fb-e5d9-4bd7-9e3e-df56563b250e None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:03.003868 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee52fb-e5d9-4bd7-9e3e-df56563b250e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:03.022456 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-a6ee52fb-e5d9-4bd7-9e3e-df56563b250e None None] MFA Rules not processed for user `cffc436199314734957ccd7232ed1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:03.023695 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-a6ee52fb-e5d9-4bd7-9e3e-df56563b250e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:03.081051 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1435/2904] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:09:03.091233 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf9709-e22f-4af5-bde4-e4431ab7aee9 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:03.092987 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-eadf9709-e22f-4af5-bde4-e4431ab7aee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:03.149626 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf9709-e22f-4af5-bde4-e4431ab7aee9 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:03.153357 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c60de-22c7-4e17-943d-20b4ba12240e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '942b54c39ae5403c8ae3bcca1c31bf1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:03.153963 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c60de-22c7-4e17-943d-20b4ba12240e service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:03.154250 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c60de-22c7-4e17-943d-20b4ba12240e service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:03.154530 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c60de-22c7-4e17-943d-20b4ba12240e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:03.164022 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c60de-22c7-4e17-943d-20b4ba12240e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:03.165417 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c60de-22c7-4e17-943d-20b4ba12240e service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:03.174762 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1470/2905] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:09:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:09:04.359393 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd58efa-b935-49a8-898a-0a5d35f1a8f0 None None] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:04.359910 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd58efa-b935-49a8-898a-0a5d35f1a8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:04.360216 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd58efa-b935-49a8-898a-0a5d35f1a8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:04.395341 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.auth.core [None req-cbd58efa-b935-49a8-898a-0a5d35f1a8f0 None None] MFA Rules not processed for user `f8b44cf61337411cb343fd4423318006`. Rule list: `[]` (Enabled: `True`). {{(pid=81844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:04.396600 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-cbd58efa-b935-49a8-898a-0a5d35f1a8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:04.461416 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1436/2906] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:09:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:09:04.465951 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c60de-22c7-4e17-943d-20b4ba12240e service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:04.475976 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c60de-22c7-4e17-943d-20b4ba12240e service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:04.480541 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56c771-e7ea-4343-aa35-8857f78f8b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:04.481131 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56c771-e7ea-4343-aa35-8857f78f8b7c admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:04.481411 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56c771-e7ea-4343-aa35-8857f78f8b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:04.481682 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56c771-e7ea-4343-aa35-8857f78f8b7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:04.482311 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf56c771-e7ea-4343-aa35-8857f78f8b7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:04.484197 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf56c771-e7ea-4343-aa35-8857f78f8b7c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:04.625596 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1471/2907] 10.222.0.70 () {60 vars in 1153 bytes} [Wed Aug 24 17:09:04 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:09:04.630031 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd58efa-b935-49a8-898a-0a5d35f1a8f0 None None] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:04.640757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd58efa-b935-49a8-898a-0a5d35f1a8f0 None None] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:04.691243 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:04.698123 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df057ca1-2879-4402-b48c-358fbba2abbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:04.698797 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df057ca1-2879-4402-b48c-358fbba2abbd admin admin] REQUEST_METHOD: `POST` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:04.699079 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df057ca1-2879-4402-b48c-358fbba2abbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:04.699348 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df057ca1-2879-4402-b48c-358fbba2abbd admin admin] PATH_INFO: `/v3/users` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:04.699980 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df057ca1-2879-4402-b48c-358fbba2abbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:04.702381 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df057ca1-2879-4402-b48c-358fbba2abbd admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:04.784628 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1437/2908] 10.222.0.70 () {60 vars in 1144 bytes} [Wed Aug 24 17:09:04 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:09:04.789361 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56c771-e7ea-4343-aa35-8857f78f8b7c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:04.799942 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56c771-e7ea-4343-aa35-8857f78f8b7c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:04.804471 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaa795-7cfc-440d-8ca6-2eec3bb0eac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:04.805293 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaa795-7cfc-440d-8ca6-2eec3bb0eac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:04.805408 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaa795-7cfc-440d-8ca6-2eec3bb0eac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:04.805634 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaa795-7cfc-440d-8ca6-2eec3bb0eac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:04.806275 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaa795-7cfc-440d-8ca6-2eec3bb0eac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:04.807862 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaa795-7cfc-440d-8ca6-2eec3bb0eac6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:04.819257 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1472/2909] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:09:04.823411 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df057ca1-2879-4402-b48c-358fbba2abbd admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:04.833292 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df057ca1-2879-4402-b48c-358fbba2abbd admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:04.838439 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71037e87-d79f-4a30-969d-e27f8e37f5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:04.839086 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71037e87-d79f-4a30-969d-e27f8e37f5d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:04.839367 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71037e87-d79f-4a30-969d-e27f8e37f5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:04.839642 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71037e87-d79f-4a30-969d-e27f8e37f5d4 admin admin] PATH_INFO: `/v3/projects/3e61471b6a1346dbb3af450d9d385366/users/1533a9a21aa54ac2a96ad570784eef75/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:04.854921 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71037e87-d79f-4a30-969d-e27f8e37f5d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e61471b6a1346dbb3af450d9d385366, user_id=1533a9a21aa54ac2a96ad570784eef75, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:04.857647 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71037e87-d79f-4a30-969d-e27f8e37f5d4 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:04.951590 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1438/2910] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:09:04 2022] PUT /identity/v3/projects/3e61471b6a1346dbb3af450d9d385366/users/1533a9a21aa54ac2a96ad570784eef75/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:09:04.955542 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaa795-7cfc-440d-8ca6-2eec3bb0eac6 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:04.962473 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:04.966255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaa795-7cfc-440d-8ca6-2eec3bb0eac6 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:05.035220 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420f2e4-0bf8-43d3-9c5e-a5c54edf43ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:05.035834 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420f2e4-0bf8-43d3-9c5e-a5c54edf43ae admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:05.036124 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420f2e4-0bf8-43d3-9c5e-a5c54edf43ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:05.036424 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420f2e4-0bf8-43d3-9c5e-a5c54edf43ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:05.037072 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420f2e4-0bf8-43d3-9c5e-a5c54edf43ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:05.039178 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420f2e4-0bf8-43d3-9c5e-a5c54edf43ae admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:05.050024 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1473/2911] 10.222.0.70 () {58 vars in 1122 bytes} [Wed Aug 24 17:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:09:05.055084 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71037e87-d79f-4a30-969d-e27f8e37f5d4 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:05.065858 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71037e87-d79f-4a30-969d-e27f8e37f5d4 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:05.070365 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67018dbe-36b6-4768-93ba-4d547f630b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:05.071004 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67018dbe-36b6-4768-93ba-4d547f630b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:05.071281 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67018dbe-36b6-4768-93ba-4d547f630b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:05.071551 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67018dbe-36b6-4768-93ba-4d547f630b15 admin admin] PATH_INFO: `/v3/projects/3e61471b6a1346dbb3af450d9d385366/users/1533a9a21aa54ac2a96ad570784eef75/roles/e7f48343fc1542979ab4459865109e30` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:05.073481 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67018dbe-36b6-4768-93ba-4d547f630b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e61471b6a1346dbb3af450d9d385366, user_id=1533a9a21aa54ac2a96ad570784eef75, role_id=e7f48343fc1542979ab4459865109e30)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:05.076096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67018dbe-36b6-4768-93ba-4d547f630b15 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:05.086247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1439/2912] 10.222.0.70 () {60 vars in 1483 bytes} [Wed Aug 24 17:09:05 2022] PUT /identity/v3/projects/3e61471b6a1346dbb3af450d9d385366/users/1533a9a21aa54ac2a96ad570784eef75/roles/e7f48343fc1542979ab4459865109e30 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:09:06.169375 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d8e25-e3ac-476d-a64d-998225b92edb None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:06.169865 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d8e25-e3ac-476d-a64d-998225b92edb None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:06.170179 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d8e25-e3ac-476d-a64d-998225b92edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:06.199255 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e41d8e25-e3ac-476d-a64d-998225b92edb None None] MFA Rules not processed for user `1533a9a21aa54ac2a96ad570784eef75`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:06.200494 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e41d8e25-e3ac-476d-a64d-998225b92edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:06.272155 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1474/2913] 10.222.0.70 () {58 vars in 958 bytes} [Wed Aug 24 17:09:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:09:06.284012 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67018dbe-36b6-4768-93ba-4d547f630b15 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:06.285757 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-67018dbe-36b6-4768-93ba-4d547f630b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:06.344198 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67018dbe-36b6-4768-93ba-4d547f630b15 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:06.347906 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402118aa-3720-4fd7-bf74-6f419d4020b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:06.348547 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402118aa-3720-4fd7-bf74-6f419d4020b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:06.348833 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402118aa-3720-4fd7-bf74-6f419d4020b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:06.349105 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402118aa-3720-4fd7-bf74-6f419d4020b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:06.404729 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402118aa-3720-4fd7-bf74-6f419d4020b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:06.406060 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402118aa-3720-4fd7-bf74-6f419d4020b8 service cinder] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:06.414687 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1440/2914] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:09:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:09:08.063234 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d8e25-e3ac-476d-a64d-998225b92edb None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:08.064698 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e41d8e25-e3ac-476d-a64d-998225b92edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:08.076383 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d8e25-e3ac-476d-a64d-998225b92edb None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:08.079803 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8712e88-e3c2-47c1-9e01-bdea43772fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f08140bcbd1e43dda77d4387666f9085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:08.080586 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8712e88-e3c2-47c1-9e01-bdea43772fbd service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:08.080867 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8712e88-e3c2-47c1-9e01-bdea43772fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:08.081137 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8712e88-e3c2-47c1-9e01-bdea43772fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:08.089695 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8712e88-e3c2-47c1-9e01-bdea43772fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:08.091090 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8712e88-e3c2-47c1-9e01-bdea43772fbd service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:08.101078 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1475/2915] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:09:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:09:16.410810 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=67,keystone:INSERT=5 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:18.096346 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=67 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:23.665412 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402118aa-3720-4fd7-bf74-6f419d4020b8 service cinder] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:23.729135 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402118aa-3720-4fd7-bf74-6f419d4020b8 service cinder] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:23.734251 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913578dc-2d60-4a30-b47b-c5182f07833c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:23.734882 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913578dc-2d60-4a30-b47b-c5182f07833c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:23.735259 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913578dc-2d60-4a30-b47b-c5182f07833c admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:23.735545 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913578dc-2d60-4a30-b47b-c5182f07833c admin admin] PATH_INFO: `/v3/users/f8b44cf61337411cb343fd4423318006` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:23.736692 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913578dc-2d60-4a30-b47b-c5182f07833c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8b44cf61337411cb343fd4423318006)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:23.738657 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913578dc-2d60-4a30-b47b-c5182f07833c admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:23.919691 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-913578dc-2d60-4a30-b47b-c5182f07833c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8b44cf61337411cb343fd4423318006'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:23.920328 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-913578dc-2d60-4a30-b47b-c5182f07833c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8b44cf61337411cb343fd4423318006'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:23.920981 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-913578dc-2d60-4a30-b47b-c5182f07833c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8b44cf61337411cb343fd4423318006'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:23.928078 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-913578dc-2d60-4a30-b47b-c5182f07833c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8b44cf61337411cb343fd4423318006'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:24.002206 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-913578dc-2d60-4a30-b47b-c5182f07833c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8b44cf61337411cb343fd4423318006'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:24.020157 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1441/2916] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:09:23 2022] DELETE /identity/v3/users/f8b44cf61337411cb343fd4423318006 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:09:24.266788 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8712e88-e3c2-47c1-9e01-bdea43772fbd service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:24.280069 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8712e88-e3c2-47c1-9e01-bdea43772fbd service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:24.337734 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:24.338365 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:24.338647 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:24.338924 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e admin admin] PATH_INFO: `/v3/projects/361a04154f264f6195120fb1ff42ba5e` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:24.340577 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=361a04154f264f6195120fb1ff42ba5e)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:24.342487 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:24.440847 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '361a04154f264f6195120fb1ff42ba5e', 'request_id': 'req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:24.446861 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '361a04154f264f6195120fb1ff42ba5e', 'request_id': 'req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:24.452363 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '361a04154f264f6195120fb1ff42ba5e', 'request_id': 'req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:24.526577 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e admin admin] The token cache is being invalidate because project 361a04154f264f6195120fb1ff42ba5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:09:24.526801 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:24.527958 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:24.529475 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1476/2917] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:09:24 2022] DELETE /identity/v3/projects/361a04154f264f6195120fb1ff42ba5e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:09:24.533837 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913578dc-2d60-4a30-b47b-c5182f07833c admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:24.535370 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-913578dc-2d60-4a30-b47b-c5182f07833c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:24.550096 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913578dc-2d60-4a30-b47b-c5182f07833c admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:24.608028 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e48b9-09e3-4974-b2dd-e1d678674f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:24.608686 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e48b9-09e3-4974-b2dd-e1d678674f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:24.608971 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e48b9-09e3-4974-b2dd-e1d678674f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:24.609247 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e48b9-09e3-4974-b2dd-e1d678674f94 admin admin] PATH_INFO: `/v3/users/1533a9a21aa54ac2a96ad570784eef75` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:24.610401 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e48b9-09e3-4974-b2dd-e1d678674f94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1533a9a21aa54ac2a96ad570784eef75)` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:24.612309 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e48b9-09e3-4974-b2dd-e1d678674f94 admin admin] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:24.633306 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:24.838948 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ab5e48b9-09e3-4974-b2dd-e1d678674f94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1533a9a21aa54ac2a96ad570784eef75'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:24.839622 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ab5e48b9-09e3-4974-b2dd-e1d678674f94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1533a9a21aa54ac2a96ad570784eef75'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:24.840742 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ab5e48b9-09e3-4974-b2dd-e1d678674f94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1533a9a21aa54ac2a96ad570784eef75'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:24.847522 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ab5e48b9-09e3-4974-b2dd-e1d678674f94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1533a9a21aa54ac2a96ad570784eef75'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:24.921429 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.notifications [None req-ab5e48b9-09e3-4974-b2dd-e1d678674f94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1533a9a21aa54ac2a96ad570784eef75'} {{(pid=81844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:24.939182 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1442/2918] 10.222.0.70 () {58 vars in 1224 bytes} [Wed Aug 24 17:09:24 2022] DELETE /identity/v3/users/1533a9a21aa54ac2a96ad570784eef75 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:09:25.192215 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:25.193780 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:25.206513 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e076eb-e8a1-4350-aa2e-1f5c28bb3e1e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:25.263828 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea01de9-b79c-4dfe-b096-c6dd1745459d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cffc436199314734957ccd7232ed1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f3b91fa60b4447a866a572209ebc37f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:25.264501 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea01de9-b79c-4dfe-b096-c6dd1745459d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:25.264789 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea01de9-b79c-4dfe-b096-c6dd1745459d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:25.265065 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea01de9-b79c-4dfe-b096-c6dd1745459d admin admin] PATH_INFO: `/v3/projects/3e61471b6a1346dbb3af450d9d385366` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:25.266183 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea01de9-b79c-4dfe-b096-c6dd1745459d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e61471b6a1346dbb3af450d9d385366)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:25.268086 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea01de9-b79c-4dfe-b096-c6dd1745459d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:25.407796 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:25.419901 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fea01de9-b79c-4dfe-b096-c6dd1745459d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e61471b6a1346dbb3af450d9d385366', 'request_id': 'req-fea01de9-b79c-4dfe-b096-c6dd1745459d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:25.426336 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fea01de9-b79c-4dfe-b096-c6dd1745459d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e61471b6a1346dbb3af450d9d385366', 'request_id': 'req-fea01de9-b79c-4dfe-b096-c6dd1745459d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:25.431988 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fea01de9-b79c-4dfe-b096-c6dd1745459d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e61471b6a1346dbb3af450d9d385366', 'request_id': 'req-fea01de9-b79c-4dfe-b096-c6dd1745459d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:25.516306 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fea01de9-b79c-4dfe-b096-c6dd1745459d admin admin] The token cache is being invalidate because project 3e61471b6a1346dbb3af450d9d385366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:09:25.516641 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fea01de9-b79c-4dfe-b096-c6dd1745459d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:25.517746 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fea01de9-b79c-4dfe-b096-c6dd1745459d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:09:25.519324 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1477/2919] 10.222.0.70 () {58 vars in 1233 bytes} [Wed Aug 24 17:09:25 2022] DELETE /identity/v3/projects/3e61471b6a1346dbb3af450d9d385366 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:09:34.935651 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:35.556313 dv-cinder-852208-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:23.358638 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e48b9-09e3-4974-b2dd-e1d678674f94 admin admin] Authenticating user token {{(pid=81844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:23.360059 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-ab5e48b9-09e3-4974-b2dd-e1d678674f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:11:23.431162 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e48b9-09e3-4974-b2dd-e1d678674f94 admin admin] Validating token access rules against request {{(pid=81844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:23.490615 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51730667-a857-4a60-963c-53dafe9148ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc4c43c85a5d49979ba803fdb1041c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc920dce30734ac7a69dc039d8c74ce1', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:23.491233 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51730667-a857-4a60-963c-53dafe9148ab service placement] REQUEST_METHOD: `GET` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:23.491556 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51730667-a857-4a60-963c-53dafe9148ab service placement] SCRIPT_NAME: `/identity` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:23.491832 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51730667-a857-4a60-963c-53dafe9148ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:23.493629 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.fernet_utils [None req-51730667-a857-4a60-963c-53dafe9148ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:11:23.504916 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51730667-a857-4a60-963c-53dafe9148ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:23.506618 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51730667-a857-4a60-963c-53dafe9148ab service placement] RBAC: Authorization granted {{(pid=81844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:23.516435 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: [pid: 81844|app: 0|req: 1443/2920] 10.222.0.70 () {60 vars in 1365 bytes} [Wed Aug 24 17:11:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:11:33.512242 dv-cinder-852208-9-iscsi devstack@keystone.service[81844]: DEBUG dbcounter [-] [81844] Writing DB stats keystone:SELECT=58 {{(pid=81844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}